[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:01:41 dist-test-slave-pww0 krb5kdc[31550](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:01:41 dist-test-slave-pww0 krb5kdc[31550](info): set up 2 sockets
Aug 07 00:01:41 dist-test-slave-pww0 krb5kdc[31550](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260807 00:01:42.618592 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.009s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:01:42 dist-test-slave-pww0 krb5kdc[31550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:01:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:01:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:01:42.750648 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41421
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:41421
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:42.870311 31565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:42.870620 31565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:42.870674 31565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:42.874758 31565 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:01:42.874832 31565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:42.874851 31565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:42.874867 31565 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:01:42.874881 31565 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:01:42.880455 31565 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:41421
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41421
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:42.881610 31565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:42.882697 31565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:01:42.891953 31565 server_base.cc:1034] running on GCE node
W20260807 00:01:42.891927 31572 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:42.891978 31574 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:42.891978 31577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:42.892849 31565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:42.894074 31565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:42.895244 31565 hybrid_clock.cc:648] HybridClock initialized: now 1786060902895263 us; error 74 us; skew 500 ppm
Aug 07 00:01:42 dist-test-slave-pww0 krb5kdc[31550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060902, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:42.898161 31565 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:01:42.899149 31565 webserver.cc:460] Webserver started at http://127.30.204.126:41883/ using document root <none> and password file <none>
I20260807 00:01:42.899768 31565 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:42.899840 31565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:42.900058 31565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:42.902313 31565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "f0e8be06a0c347e2935da9f6ac38b6c5"
format_stamp: "Formatted at 2026-08-07 00:01:42 on dist-test-slave-pww0"
server_key: "de2c64426b9837bd792a957a47ce8c5b"
server_key_iv: "a1cd06e314a704e5823095fd4f8f62f5"
server_key_version: "encryptionkey@0"
I20260807 00:01:42.902912 31565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "f0e8be06a0c347e2935da9f6ac38b6c5"
format_stamp: "Formatted at 2026-08-07 00:01:42 on dist-test-slave-pww0"
server_key: "de2c64426b9837bd792a957a47ce8c5b"
server_key_iv: "a1cd06e314a704e5823095fd4f8f62f5"
server_key_version: "encryptionkey@0"
I20260807 00:01:42.906843 31565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:01:42.909518 31585 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:42.910815 31565 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 00:01:42.910941 31565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "f0e8be06a0c347e2935da9f6ac38b6c5"
format_stamp: "Formatted at 2026-08-07 00:01:42 on dist-test-slave-pww0"
server_key: "de2c64426b9837bd792a957a47ce8c5b"
server_key_iv: "a1cd06e314a704e5823095fd4f8f62f5"
server_key_version: "encryptionkey@0"
I20260807 00:01:42.911049 31565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:42.928735 31565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:42.932081 31565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:42.932286 31565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:42.939697 31565 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:41421
I20260807 00:01:42.939718 31673 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:41421 every 8 connection(s)
I20260807 00:01:42.940730 31565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:01:42.943661 31675 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:42.949175 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31565
I20260807 00:01:42.949358 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:01:42.950125 31675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5: Bootstrap starting.
I20260807 00:01:42.951112 31537 external_mini_cluster.cc:1428] Setting key f4064e6841b21d975300bf506de4a671
I20260807 00:01:42.953507 31675 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:42.954404 31675 log.cc:824] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:42.956634 31675 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5: No bootstrap required, opened a new log
I20260807 00:01:42.960928 31675 raft_consensus.cc:348] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41421 } }
I20260807 00:01:42.961236 31675 raft_consensus.cc:374] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:42.961336 31675 raft_consensus.cc:729] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f0e8be06a0c347e2935da9f6ac38b6c5, State: Initialized, Role: FOLLOWER
I20260807 00:01:42.961833 31675 consensus_queue.cc:260] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41421 } }
I20260807 00:01:42.961970 31675 raft_consensus.cc:388] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:01:42.962034 31675 raft_consensus.cc:482] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:01:42.962134 31675 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:42.963107 31675 raft_consensus.cc:504] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41421 } }
I20260807 00:01:42.963441 31675 leader_election.cc:302] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f0e8be06a0c347e2935da9f6ac38b6c5; no voters:
I20260807 00:01:42.963815 31675 leader_election.cc:288] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:01:42.963943 31683 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:42.964174 31683 raft_consensus.cc:686] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [term 1 LEADER]: Becoming Leader. State: Replica: f0e8be06a0c347e2935da9f6ac38b6c5, State: Running, Role: LEADER
I20260807 00:01:42.964535 31683 consensus_queue.cc:237] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41421 } }
I20260807 00:01:42.965137 31675 sys_catalog.cc:564] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:01:42.966967 31684 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41421 } } }
I20260807 00:01:42.967144 31684 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:42.967545 31685 sys_catalog.cc:455] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f0e8be06a0c347e2935da9f6ac38b6c5. Latest consensus state: current_term: 1 leader_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f0e8be06a0c347e2935da9f6ac38b6c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41421 } } }
I20260807 00:01:42.967655 31685 sys_catalog.cc:458] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5 [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:42.968088 31697 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:01:42.971516 31697 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Aug 07 00:01:42 dist-test-slave-pww0 krb5kdc[31550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060902, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:42.978789 31697 catalog_manager.cc:1347] Generated new cluster ID: c6c45ad546ef4cb1917df4c86d1fb977
I20260807 00:01:42.978865 31697 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:01:42.981912 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:42.954880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51804 (local address 127.30.204.126:41421)
0807 00:01:42.955385 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:42.955406 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:42.956697 (+ 1291us) server_negotiation.cc:407] Connection header received
0807 00:01:42.960403 (+ 3706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:42.960430 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:42.960855 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:42.961276 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:42.963758 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:42.965216 (+ 1458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:42.966943 (+ 1727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:42.968245 (+ 1302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:42.968411 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:42.973525 (+ 5114us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:42.973531 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:42.973546 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:42.973582 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:42.976599 (+ 3017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:42.977493 (+ 894us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:42.977501 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:42.977508 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:42.977707 (+ 199us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:42.978165 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:42.978168 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:42.978170 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:42.978894 (+ 724us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:42.979116 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:42.980160 (+ 1044us) server_negotiation.cc:299] Negotiation successful
0807 00:01:42.980515 (+ 355us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":157,"threads_started":1}
I20260807 00:01:42.996047 31697 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:01:42.997040 31697 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:01:43.005829 31697 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f0e8be06a0c347e2935da9f6ac38b6c5: Generated new TSK 0
I20260807 00:01:43.006806 31697 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:01:43.047251 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41421
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:43.167982 31722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:43.168342 31722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:43.168390 31722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:43.172930 31722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:43.173031 31722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:43.173105 31722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:01:43.179211 31722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:43.180469 31722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:43.181594 31722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:43.191807 31731 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:43.191812 31732 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:43.191922 31722 server_base.cc:1034] running on GCE node
W20260807 00:01:43.191835 31734 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:43.192543 31722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:43.193219 31722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:43.194396 31722 hybrid_clock.cc:648] HybridClock initialized: now 1786060903194369 us; error 42 us; skew 500 ppm
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:43.197392 31722 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:01:43.198362 31722 webserver.cc:460] Webserver started at http://127.30.204.65:34195/ using document root <none> and password file <none>
I20260807 00:01:43.198967 31722 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:43.199041 31722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:43.199266 31722 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:43.201524 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "e4044b30e0fc48c087080e469259a8b4"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "7e1306d045b47850f15816c596893205"
server_key_iv: "9b5452fbee16b8b4b882437df3b93580"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.202142 31722 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "e4044b30e0fc48c087080e469259a8b4"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "7e1306d045b47850f15816c596893205"
server_key_iv: "9b5452fbee16b8b4b882437df3b93580"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.206223 31722 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:01:43.208927 31747 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.210294 31722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:01:43.210415 31722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "e4044b30e0fc48c087080e469259a8b4"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "7e1306d045b47850f15816c596893205"
server_key_iv: "9b5452fbee16b8b4b882437df3b93580"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.210523 31722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:43.224721 31722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:43.228320 31722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:43.228529 31722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:43.229198 31722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:43.230093 31722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:43.230155 31722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.230229 31722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:43.230254 31722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.239070 31722 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:36229
I20260807 00:01:43.239094 31923 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:36229 every 8 connection(s)
I20260807 00:01:43.240104 31722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:01:43.245487 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31722
I20260807 00:01:43.245591 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:01:43.245895 31537 external_mini_cluster.cc:1428] Setting key 54392cfa6f9e527adb723cefbca3182f
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:43.255322 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.242125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:37153 (local address 127.30.204.126:41421)
0807 00:01:43.242279 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:43.242283 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:43.243384 (+ 1101us) server_negotiation.cc:407] Connection header received
0807 00:01:43.244305 (+ 921us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:43.244307 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:43.244359 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:43.244437 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:43.245754 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.246482 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.247865 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.248099 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.248186 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.251362 (+ 3176us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:43.251366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:43.251368 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:43.251397 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:43.252870 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.253458 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.253462 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.253463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.253550 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.253917 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.253920 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.253922 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.254129 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:43.254220 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:43.254909 (+ 689us) server_negotiation.cc:299] Negotiation successful
0807 00:01:43.255122 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:01:43.256387 31928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.242528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41421 (local address 127.30.204.65:37153)
0807 00:01:43.243218 (+ 690us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:43.243264 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:43.244109 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:43.244601 (+ 492us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:43.244607 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:43.245008 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:43.245526 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:43.245541 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.246627 (+ 1086us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.246631 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:43.247741 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:43.247756 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.248273 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.248277 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:43.248496 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.249161 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:43.249185 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:43.251135 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:43.253019 (+ 1884us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:43.253027 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:43.253047 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:43.253344 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:43.253652 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:43.253655 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:43.253657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:43.253823 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:43.254241 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:43.254249 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:43.254530 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 00:01:43.254859 (+ 329us) client_negotiation.cc:240] Negotiation successful
0807 00:01:43.255271 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":475,"thread_start_us":114,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:01:43.257848 31925 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41421
I20260807 00:01:43.258112 31925 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:43.258678 31925 heartbeater.cc:511] Master 127.30.204.126:41421 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:01:43.260500 31621 ts_manager.cc:194] Registered new tserver with Master: e4044b30e0fc48c087080e469259a8b4 (127.30.204.65:36229)
I20260807 00:01:43.261826 31621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:37153
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:01:43.278118 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41421
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:43.396886 31932 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:43.397171 31932 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:43.397217 31932 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:43.401360 31932 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:43.401440 31932 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:43.401502 31932 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:01:43.407088 31932 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31932
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:43.408283 31932 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:43.409394 31932 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:43.419597 31941 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:43.419597 31942 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:43.419613 31945 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:43.420053 31932 server_base.cc:1034] running on GCE node
I20260807 00:01:43.420492 31932 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:43.421206 31932 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:43.422413 31932 hybrid_clock.cc:648] HybridClock initialized: now 1786060903422386 us; error 39 us; skew 500 ppm
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:43.425390 31932 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:01:43.426375 31932 webserver.cc:460] Webserver started at http://127.30.204.66:45125/ using document root <none> and password file <none>
I20260807 00:01:43.426986 31932 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:43.427058 31932 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:43.427291 31932 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:43.429495 31932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "0f6ae2f4009a4aac9fa8fed3c00542ab"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "122dbc3ce026a93bf99ca11d3611870f"
server_key_iv: "a9edf6d8392fab6b6e7708b5e255b1f7"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.430109 31932 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "0f6ae2f4009a4aac9fa8fed3c00542ab"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "122dbc3ce026a93bf99ca11d3611870f"
server_key_iv: "a9edf6d8392fab6b6e7708b5e255b1f7"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.434187 31932 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 00:01:43.436866 31954 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.438333 31932 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:01:43.438478 31932 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "0f6ae2f4009a4aac9fa8fed3c00542ab"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "122dbc3ce026a93bf99ca11d3611870f"
server_key_iv: "a9edf6d8392fab6b6e7708b5e255b1f7"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.438601 31932 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:43.455114 31932 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:43.458696 31932 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:43.458914 31932 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:43.459642 31932 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:43.460546 31932 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:43.460598 31932 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.460656 31932 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:43.460686 31932 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.470108 31932 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:38197
I20260807 00:01:43.470139 32124 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:38197 every 8 connection(s)
I20260807 00:01:43.471132 31932 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:01:43.476244 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31932
I20260807 00:01:43.476363 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:01:43.476677 31537 external_mini_cluster.cc:1428] Setting key 38079616ca0c8311d3b68b371c3bad25
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:43.486199 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.473092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:57131 (local address 127.30.204.126:41421)
0807 00:01:43.473273 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:43.473278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:43.474421 (+ 1143us) server_negotiation.cc:407] Connection header received
0807 00:01:43.475396 (+ 975us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:43.475399 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:43.475493 (+ 94us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:43.475579 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:43.476968 (+ 1389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.477554 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.478937 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.479275 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.479376 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.482135 (+ 2759us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:43.482138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:43.482140 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:43.482167 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:43.483623 (+ 1456us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.484326 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.484330 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.484331 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.484424 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.484863 (+ 439us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.484866 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.484867 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.485074 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:43.485158 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:43.485896 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 00:01:43.486062 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 00:01:43.487419 32128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.473557 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41421 (local address 127.30.204.66:57131)
0807 00:01:43.474258 (+ 701us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:43.474303 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:43.475124 (+ 821us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:43.475776 (+ 652us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:43.475783 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:43.476215 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:43.476811 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:43.476828 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.477696 (+ 868us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.477699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:43.478804 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:43.478816 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.479423 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.479426 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:43.479635 (+ 209us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.480237 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:43.480256 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:43.481933 (+ 1677us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:43.483761 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:43.483768 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:43.483788 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:43.484191 (+ 403us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:43.484522 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:43.484525 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:43.484527 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:43.484744 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:43.485185 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:43.485194 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:43.485571 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 00:01:43.485806 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 00:01:43.486245 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":493,"thread_start_us":128,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:01:43.488785 32126 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41421
I20260807 00:01:43.489071 32126 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:43.489639 32126 heartbeater.cc:511] Master 127.30.204.126:41421 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:01:43.490931 31621 ts_manager.cc:194] Registered new tserver with Master: 0f6ae2f4009a4aac9fa8fed3c00542ab (127.30.204.66:38197)
I20260807 00:01:43.491604 31621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:57131
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:01:43.507503 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41421
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42391
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:43.625234 32132 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:43.625524 32132 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:43.625569 32132 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:43.629740 32132 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:43.629822 32132 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:43.629884 32132 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:01:43.635388 32132 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42391
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41421
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32132
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:43.636588 32132 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:43.637709 32132 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:43.648159 32144 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:43.648255 32132 server_base.cc:1034] running on GCE node
W20260807 00:01:43.648159 32140 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:43.648151 32141 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:43.648965 32132 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:43.649721 32132 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:43.650954 32132 hybrid_clock.cc:648] HybridClock initialized: now 1786060903650929 us; error 38 us; skew 500 ppm
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:43.654130 32132 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:01:43.655203 32132 webserver.cc:460] Webserver started at http://127.30.204.67:38241/ using document root <none> and password file <none>
I20260807 00:01:43.655843 32132 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:43.655910 32132 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:43.656090 32132 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:43.658303 32132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "18095279bf48458591d46ae7a0411a89"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "a0593d4becb95a813dbed49e0ca73a39"
server_key_iv: "1de05613d50f144eca740e4b7d868c4d"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.658926 32132 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "18095279bf48458591d46ae7a0411a89"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "a0593d4becb95a813dbed49e0ca73a39"
server_key_iv: "1de05613d50f144eca740e4b7d868c4d"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.663085 32132 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:01:43.666028 32163 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.667500 32132 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:43.667629 32132 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "18095279bf48458591d46ae7a0411a89"
format_stamp: "Formatted at 2026-08-07 00:01:43 on dist-test-slave-pww0"
server_key: "a0593d4becb95a813dbed49e0ca73a39"
server_key_iv: "1de05613d50f144eca740e4b7d868c4d"
server_key_version: "encryptionkey@0"
I20260807 00:01:43.667758 32132 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:43.686178 32132 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:43.689738 32132 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:43.689913 32132 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:43.690569 32132 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:43.691608 32132 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:43.691668 32132 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.691712 32132 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:43.691732 32132 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:43.700917 32132 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:36073
I20260807 00:01:43.700943 32328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:36073 every 8 connection(s)
I20260807 00:01:43.701901 32132 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:01:43.705463 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 32132
I20260807 00:01:43.705596 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:01:43.705900 31537 external_mini_cluster.cc:1428] Setting key 8a731761c69370ab1794feb4268d1013
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:43.716967 31680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.704095 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:40677 (local address 127.30.204.126:41421)
0807 00:01:43.704290 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:43.704294 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:43.705456 (+ 1162us) server_negotiation.cc:407] Connection header received
0807 00:01:43.706651 (+ 1195us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:43.706655 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:43.706713 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:43.706786 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:43.708330 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.709026 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.710217 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.710458 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.710552 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.713299 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:43.713302 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:43.713304 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:43.713334 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:43.714785 (+ 1451us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.715337 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.715341 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.715342 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.715429 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.715811 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.715814 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.715816 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.716013 (+ 197us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:43.716102 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:43.716657 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 00:01:43.716797 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 00:01:43.717840 32332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.704479 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41421 (local address 127.30.204.67:40677)
0807 00:01:43.705234 (+ 755us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:43.705316 (+ 82us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:43.706376 (+ 1060us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:43.706992 (+ 616us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:43.707002 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:43.707569 (+ 567us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:43.708185 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:43.708200 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.709162 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.709166 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:43.710079 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:43.710089 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.710625 (+ 536us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.710630 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:43.710817 (+ 187us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.711430 (+ 613us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:43.711449 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:43.713100 (+ 1651us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:43.714910 (+ 1810us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:43.714917 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:43.714937 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:43.715236 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:43.715536 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:43.715540 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:43.715542 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:43.715713 (+ 171us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:43.716124 (+ 411us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:43.716131 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:43.716404 (+ 273us) client_negotiation.cc:769] Sending connection context
0807 00:01:43.716587 (+ 183us) client_negotiation.cc:240] Negotiation successful
0807 00:01:43.716948 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":526,"thread_start_us":129,"threads_started":1}
I20260807 00:01:43.719033 32330 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41421
I20260807 00:01:43.719304 32330 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:43.719965 32330 heartbeater.cc:511] Master 127.30.204.126:41421 requested a full tablet report, sending...
I20260807 00:01:43.721149 31621 ts_manager.cc:194] Registered new tserver with Master: 18095279bf48458591d46ae7a0411a89 (127.30.204.67:36073)
I20260807 00:01:43.721731 31621 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:40677
I20260807 00:01:43.733038 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:01:43.762576 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.752709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33418 (local address 127.30.204.65:36229)
0807 00:01:43.753038 (+ 329us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:43.753052 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:43.753079 (+ 27us) server_negotiation.cc:407] Connection header received
0807 00:01:43.753153 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:43.753159 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:43.753359 (+ 200us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:43.753481 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:43.754279 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.755108 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.756121 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.756523 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.756633 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.759095 (+ 2462us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:43.759109 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:43.759125 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:43.759155 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:43.760766 (+ 1611us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.761216 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.761221 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.761223 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.761306 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.761628 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.761630 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.761631 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.761912 (+ 281us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:43.762078 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:43.762243 (+ 165us) server_negotiation.cc:299] Negotiation successful
0807 00:01:43.762381 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":213,"thread_start_us":103,"threads_started":1}
W20260807 00:01:43.763706 31795 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33418
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[31550](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060903, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:01:43.791396 32339 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:43.783215 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33432 (local address 127.30.204.65:36229)
0807 00:01:43.783380 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:43.783386 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:43.783493 (+ 107us) server_negotiation.cc:407] Connection header received
0807 00:01:43.783664 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:43.783669 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:43.783717 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:43.783786 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:43.784500 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.785015 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.785863 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:43.786089 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:43.786182 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:43.788604 (+ 2422us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:43.788609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:43.788611 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:43.788673 (+ 62us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:43.789847 (+ 1174us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.790253 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.790255 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.790256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.790336 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:43.790593 (+ 257us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:43.790595 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:43.790596 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:43.790781 (+ 185us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:43.790899 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:43.791106 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 00:01:43.791228 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:01:43.792557 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31722
I20260807 00:01:43.808389 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31932
I20260807 00:01:43.825685 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 32132
I20260807 00:01:43.838006 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31565
2026-08-07T00:01:43Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (2396 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[32350](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[32350](info): set up 2 sockets
Aug 07 00:01:43 dist-test-slave-pww0 krb5kdc[32350](info): commencing operation
krb5kdc: starting...
W20260807 00:01:45.001969 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.136s user 0.004s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:01:45 dist-test-slave-pww0 krb5kdc[32350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:01:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:01:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:01:45.138545 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40931
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:40931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:45.256821 32373 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:45.257144 32373 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:45.257205 32373 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:45.261747 32373 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:01:45.261829 32373 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:45.261858 32373 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:45.261880 32373 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:01:45.261910 32373 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:01:45.267608 32373 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:40931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32373
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:45.268822 32373 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:45.269950 32373 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:45.278993 32384 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:45.278954 32382 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:45.278982 32373 server_base.cc:1034] running on GCE node
W20260807 00:01:45.278982 32381 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:45.279850 32373 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:45.281090 32373 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:45.282279 32373 hybrid_clock.cc:648] HybridClock initialized: now 1786060905282258 us; error 36 us; skew 500 ppm
Aug 07 00:01:45 dist-test-slave-pww0 krb5kdc[32350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:45.285321 32373 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:01:45.286345 32373 webserver.cc:460] Webserver started at http://127.30.204.126:37665/ using document root <none> and password file <none>
I20260807 00:01:45.286945 32373 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:45.287017 32373 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:45.287248 32373 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:45.289453 32373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "53848be277bc70cfa8165881b506c20c"
server_key_iv: "8eea09605396a5961db147c1b54871b3"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.290045 32373 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "53848be277bc70cfa8165881b506c20c"
server_key_iv: "8eea09605396a5961db147c1b54871b3"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.294116 32373 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:01:45.296947 32396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:45.298327 32373 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:01:45.298465 32373 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "53848be277bc70cfa8165881b506c20c"
server_key_iv: "8eea09605396a5961db147c1b54871b3"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.298583 32373 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:45.317660 32373 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:45.321300 32373 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:45.321524 32373 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:45.329399 32373 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:40931
I20260807 00:01:45.329468 32485 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:40931 every 8 connection(s)
I20260807 00:01:45.330502 32373 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:01:45.333636 32487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:45.336385 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 32373
I20260807 00:01:45.336494 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:01:45.336735 31537 external_mini_cluster.cc:1428] Setting key 79aea1c85d965ae5823c72ab9f2ce826
I20260807 00:01:45.340431 32487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d: Bootstrap starting.
I20260807 00:01:45.343295 32487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:45.344229 32487 log.cc:824] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:45.346695 32487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d: No bootstrap required, opened a new log
Aug 07 00:01:45 dist-test-slave-pww0 krb5kdc[32350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:45.350217 32487 raft_consensus.cc:348] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40931 } }
I20260807 00:01:45.350484 32487 raft_consensus.cc:374] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:45.350528 32487 raft_consensus.cc:729] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bce3ba2ca046ed9b1fea7ebda0a30d, State: Initialized, Role: FOLLOWER
I20260807 00:01:45.351046 32487 consensus_queue.cc:260] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40931 } }
I20260807 00:01:45.351222 32487 raft_consensus.cc:388] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:01:45.351295 32487 raft_consensus.cc:482] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:01:45.351397 32487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:45.352654 32487 raft_consensus.cc:504] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40931 } }
I20260807 00:01:45.353049 32487 leader_election.cc:302] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e2bce3ba2ca046ed9b1fea7ebda0a30d; no voters:
I20260807 00:01:45.353413 32487 leader_election.cc:288] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:01:45.353547 32494 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:45.353569 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:45.338257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35956 (local address 127.30.204.126:40931)
0807 00:01:45.338735 (+ 478us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:45.338756 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:45.338803 (+ 47us) server_negotiation.cc:407] Connection header received
0807 00:01:45.339689 (+ 886us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:45.339715 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:45.340180 (+ 465us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:45.340580 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:45.341415 (+ 835us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.342903 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.343870 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.344597 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.344761 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:45.347699 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:45.347704 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:45.347720 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:45.347752 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:45.350064 (+ 2312us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:45.350597 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:45.350604 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:45.350611 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:45.350774 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:45.351063 (+ 289us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:45.351066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:45.351068 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:45.351693 (+ 625us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:45.351901 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:45.352148 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 00:01:45.352481 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":110,"threads_started":1}
I20260807 00:01:45.353845 32494 raft_consensus.cc:686] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [term 1 LEADER]: Becoming Leader. State: Replica: e2bce3ba2ca046ed9b1fea7ebda0a30d, State: Running, Role: LEADER
I20260807 00:01:45.354213 32494 consensus_queue.cc:237] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40931 } }
I20260807 00:01:45.354784 32487 sys_catalog.cc:564] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:01:45.356590 32496 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40931 } } }
I20260807 00:01:45.356725 32496 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:45.357195 32498 sys_catalog.cc:455] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [sys.catalog]: SysCatalogTable state changed. Reason: New leader e2bce3ba2ca046ed9b1fea7ebda0a30d. Latest consensus state: current_term: 1 leader_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e2bce3ba2ca046ed9b1fea7ebda0a30d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40931 } } }
I20260807 00:01:45.357410 32498 sys_catalog.cc:458] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d [sys.catalog]: This master's current role is: LEADER
W20260807 00:01:45.360013 32520 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:01:45.360093 32520 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:01:45.360169 32521 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:01:45.360921 32521 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:01:45.367126 32521 catalog_manager.cc:1347] Generated new cluster ID: 1ff2a8e477d047e28c91bf5759853fb5
I20260807 00:01:45.367218 32521 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:01:45.375829 32521 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:01:45.377286 32521 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:01:45.387050 32521 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e2bce3ba2ca046ed9b1fea7ebda0a30d: Generated new TSK 0
I20260807 00:01:45.388063 32521 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:01:45.422662 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40931
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:45.540673 32528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:45.540983 32528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:45.541030 32528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:45.545284 32528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:45.545368 32528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:45.545431 32528 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:01:45.551110 32528 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40931
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:45.552358 32528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:45.553505 32528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:45.564126 32538 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:45.564127 32539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:45.564131 32543 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:45.564194 32528 server_base.cc:1034] running on GCE node
I20260807 00:01:45.564909 32528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:45.565621 32528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:45.566818 32528 hybrid_clock.cc:648] HybridClock initialized: now 1786060905566767 us; error 68 us; skew 500 ppm
Aug 07 00:01:45 dist-test-slave-pww0 krb5kdc[32350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:45.570139 32528 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:01:45.571220 32528 webserver.cc:460] Webserver started at http://127.30.204.65:45879/ using document root <none> and password file <none>
I20260807 00:01:45.571930 32528 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:45.572039 32528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:45.572296 32528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:45.574962 32528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "90e3bd31d16b41c088cc2c4d61668c31"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "e255c43a8bb916729b8df43aa9c6dc21"
server_key_iv: "a58ca1e784ba3e25da695b7268c3551b"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.575712 32528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "90e3bd31d16b41c088cc2c4d61668c31"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "e255c43a8bb916729b8df43aa9c6dc21"
server_key_iv: "a58ca1e784ba3e25da695b7268c3551b"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.580163 32528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:01:45.583175 32553 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:45.584623 32528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:45.584767 32528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "90e3bd31d16b41c088cc2c4d61668c31"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "e255c43a8bb916729b8df43aa9c6dc21"
server_key_iv: "a58ca1e784ba3e25da695b7268c3551b"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.584887 32528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:45.597049 32528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:45.600553 32528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:45.600760 32528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:45.601411 32528 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:45.602348 32528 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:45.602398 32528 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:45.602456 32528 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:45.602483 32528 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:45.611506 32528 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:38297
I20260807 00:01:45.611536 32759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:38297 every 8 connection(s)
I20260807 00:01:45.612509 32528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:01:45.620671 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 32528
I20260807 00:01:45.620774 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:01:45.621049 31537 external_mini_cluster.cc:1428] Setting key c87fee10a1933c58b1a7de1083ecf60b
Aug 07 00:01:45 dist-test-slave-pww0 krb5kdc[32350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:45.627733 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:45.614674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:46223 (local address 127.30.204.126:40931)
0807 00:01:45.614849 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:45.614853 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:45.615748 (+ 895us) server_negotiation.cc:407] Connection header received
0807 00:01:45.617295 (+ 1547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:45.617298 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:45.617362 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:45.617437 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:45.618711 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.619304 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.620572 (+ 1268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.620865 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.620962 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:45.624120 (+ 3158us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:45.624123 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:45.624125 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:45.624151 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:45.625489 (+ 1338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:45.626053 (+ 564us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:45.626056 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:45.626057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:45.626144 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:45.626503 (+ 359us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:45.626506 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:45.626507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:45.626710 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:45.626799 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:45.627374 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 00:01:45.627559 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260807 00:01:45.628643 32762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:45.615019 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40931 (local address 127.30.204.65:46223)
0807 00:01:45.615562 (+ 543us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:45.615637 (+ 75us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:45.617056 (+ 1419us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:45.617648 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:45.617655 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:45.618035 (+ 380us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:45.618555 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:45.618569 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.619436 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.619439 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:45.620407 (+ 968us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:45.620420 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.621059 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.621064 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:45.621310 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:45.622006 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:45.622027 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:45.623929 (+ 1902us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:45.625627 (+ 1698us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:45.625634 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:45.625654 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:45.625937 (+ 283us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:45.626230 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:45.626233 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:45.626235 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:45.626400 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:45.626815 (+ 415us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:45.626823 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:45.627114 (+ 291us) client_negotiation.cc:769] Sending connection context
0807 00:01:45.627295 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 00:01:45.627707 (+ 412us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":127,"threads_started":1}
I20260807 00:01:45.630043 32760 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40931
I20260807 00:01:45.630311 32760 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:45.630872 32760 heartbeater.cc:511] Master 127.30.204.126:40931 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:01:45.632619 32419 ts_manager.cc:194] Registered new tserver with Master: 90e3bd31d16b41c088cc2c4d61668c31 (127.30.204.65:38297)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:01:45.634260 32419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:46223
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:01:45.651204 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40931
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:45.769610 302 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:45.769907 302 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:45.769954 302 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:45.774278 302 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:45.774364 302 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:45.774430 302 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:01:45.780248 302 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40931
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.302
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:45.781540 302 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:45.782673 302 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:45.793293 317 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:45.793327 311 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:45.793293 308 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:45.793375 302 server_base.cc:1034] running on GCE node
I20260807 00:01:45.793990 302 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:45.794696 302 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:45.795869 302 hybrid_clock.cc:648] HybridClock initialized: now 1786060905795847 us; error 41 us; skew 500 ppm
Aug 07 00:01:45 dist-test-slave-pww0 krb5kdc[32350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:45.798852 302 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:01:45.800007 302 webserver.cc:460] Webserver started at http://127.30.204.66:35183/ using document root <none> and password file <none>
I20260807 00:01:45.800619 302 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:45.800696 302 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:45.800920 302 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:45.803184 302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "a11326f5d79c4f24b4ab698f3afcb486"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "d4d6e5ccfc74e187cc0e421c155e0e6b"
server_key_iv: "5888a53e34d077a0f924a9fa8e60f76c"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.803879 302 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "a11326f5d79c4f24b4ab698f3afcb486"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "d4d6e5ccfc74e187cc0e421c155e0e6b"
server_key_iv: "5888a53e34d077a0f924a9fa8e60f76c"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.808027 302 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:01:45.810758 331 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:45.812107 302 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:45.812227 302 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "a11326f5d79c4f24b4ab698f3afcb486"
format_stamp: "Formatted at 2026-08-07 00:01:45 on dist-test-slave-pww0"
server_key: "d4d6e5ccfc74e187cc0e421c155e0e6b"
server_key_iv: "5888a53e34d077a0f924a9fa8e60f76c"
server_key_version: "encryptionkey@0"
I20260807 00:01:45.812350 302 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:45.845949 302 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:45.849242 302 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:45.849444 302 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:45.850096 302 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:45.851022 302 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:45.851073 302 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:45.851114 302 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:45.851142 302 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:45.860493 302 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:37907
I20260807 00:01:45.860519 516 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:37907 every 8 connection(s)
I20260807 00:01:45.861478 302 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:01:45.869707 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 302
I20260807 00:01:45.869854 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:01:45.870117 31537 external_mini_cluster.cc:1428] Setting key fefccfe6d65ecbade62468363f742441
Aug 07 00:01:45 dist-test-slave-pww0 krb5kdc[32350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:45.876820 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:45.863502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:37927 (local address 127.30.204.126:40931)
0807 00:01:45.863644 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:45.863648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:45.864584 (+ 936us) server_negotiation.cc:407] Connection header received
0807 00:01:45.866020 (+ 1436us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:45.866023 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:45.866089 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:45.866167 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:45.867543 (+ 1376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.868083 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.869319 (+ 1236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.869620 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.869742 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:45.872671 (+ 2929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:45.872675 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:45.872678 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:45.872712 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:45.874267 (+ 1555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:45.874981 (+ 714us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:45.874984 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:45.874985 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:45.875077 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:45.875508 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:45.875511 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:45.875512 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:45.875733 (+ 221us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:45.875828 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:45.876488 (+ 660us) server_negotiation.cc:299] Negotiation successful
0807 00:01:45.876648 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 00:01:45.877847 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:45.863931 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40931 (local address 127.30.204.66:37927)
0807 00:01:45.864424 (+ 493us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:45.864476 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:45.865787 (+ 1311us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:45.866378 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:45.866385 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:45.866792 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:45.867325 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:45.867341 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.868220 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.868223 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:45.869174 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:45.869186 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:45.869833 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:45.869839 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:45.870039 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:45.870673 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:45.870697 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:45.872471 (+ 1774us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:45.874406 (+ 1935us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:45.874415 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:45.874437 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:45.874851 (+ 414us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:45.875167 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:45.875170 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:45.875172 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:45.875368 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:45.875844 (+ 476us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:45.875871 (+ 27us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:45.876225 (+ 354us) client_negotiation.cc:769] Sending connection context
0807 00:01:45.876419 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 00:01:45.876798 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"thread_start_us":117,"threads_started":1}
I20260807 00:01:45.879081 517 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40931
I20260807 00:01:45.879336 517 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:45.879920 517 heartbeater.cc:511] Master 127.30.204.126:40931 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:01:45.881062 32419 ts_manager.cc:194] Registered new tserver with Master: a11326f5d79c4f24b4ab698f3afcb486 (127.30.204.66:37907)
I20260807 00:01:45.881716 32419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:37927
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:01:45.900722 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40931
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41175
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:46.019191 526 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:46.019522 526 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:46.019590 526 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:46.023838 526 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:46.023919 526 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:46.024006 526 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:01:46.029726 526 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41175
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40931
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.526
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:46.030934 526 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:46.032073 526 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:46.042766 535 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:46.042739 539 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:46.042748 542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:46.043123 526 server_base.cc:1034] running on GCE node
I20260807 00:01:46.043746 526 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:46.044486 526 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:46.045683 526 hybrid_clock.cc:648] HybridClock initialized: now 1786060906045661 us; error 40 us; skew 500 ppm
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[32350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:46.048827 526 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:01:46.049873 526 webserver.cc:460] Webserver started at http://127.30.204.67:37199/ using document root <none> and password file <none>
I20260807 00:01:46.050470 526 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:46.050544 526 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:46.050770 526 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:46.053009 526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "262523c20a6041cc96959e2534ae28c5"
format_stamp: "Formatted at 2026-08-07 00:01:46 on dist-test-slave-pww0"
server_key: "d15f245a512a8ddc9d79465dbbb0ef0d"
server_key_iv: "e4c977fbc46ce53788c67691f1ed6af7"
server_key_version: "encryptionkey@0"
I20260807 00:01:46.053679 526 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "262523c20a6041cc96959e2534ae28c5"
format_stamp: "Formatted at 2026-08-07 00:01:46 on dist-test-slave-pww0"
server_key: "d15f245a512a8ddc9d79465dbbb0ef0d"
server_key_iv: "e4c977fbc46ce53788c67691f1ed6af7"
server_key_version: "encryptionkey@0"
I20260807 00:01:46.058063 526 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:01:46.061043 552 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:46.062479 526 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:01:46.062651 526 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "262523c20a6041cc96959e2534ae28c5"
format_stamp: "Formatted at 2026-08-07 00:01:46 on dist-test-slave-pww0"
server_key: "d15f245a512a8ddc9d79465dbbb0ef0d"
server_key_iv: "e4c977fbc46ce53788c67691f1ed6af7"
server_key_version: "encryptionkey@0"
I20260807 00:01:46.062778 526 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:46.080832 526 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:46.084685 526 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:46.084897 526 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:46.085592 526 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:46.086534 526 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:46.086594 526 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:46.086652 526 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:46.086678 526 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:46.096035 526 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:36741
I20260807 00:01:46.096062 745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:36741 every 8 connection(s)
I20260807 00:01:46.097046 526 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:01:46.098834 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 526
I20260807 00:01:46.098958 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:01:46.099253 31537 external_mini_cluster.cc:1428] Setting key fb750e707b00a7f6b7536c77919ac527
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[32350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060906, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:46.112721 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.099006 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:33251 (local address 127.30.204.126:40931)
0807 00:01:46.099187 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.099193 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.100320 (+ 1127us) server_negotiation.cc:407] Connection header received
0807 00:01:46.102001 (+ 1681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.102004 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.102059 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.102136 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:46.103423 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.104034 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.105305 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.105548 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.105650 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.108474 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:46.108478 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:46.108480 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:46.108511 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:46.110035 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.110748 (+ 713us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.110751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.110753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.110853 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.111278 (+ 425us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.111281 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.111282 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.111580 (+ 298us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:46.111671 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.112399 (+ 728us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.112559 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 00:01:46.113927 752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.099500 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40931 (local address 127.30.204.67:33251)
0807 00:01:46.100161 (+ 661us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:46.100228 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:46.101749 (+ 1521us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:46.102305 (+ 556us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:46.102311 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:46.102751 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:46.103289 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.103304 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.104196 (+ 892us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.104199 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.105163 (+ 964us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.105174 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.105745 (+ 571us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.105748 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.105919 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.106552 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:46.106572 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:46.108242 (+ 1670us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:46.110209 (+ 1967us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:46.110217 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:46.110246 (+ 29us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:46.110609 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:46.110968 (+ 359us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:46.110971 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:46.110973 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:46.111155 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:46.111686 (+ 531us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:46.111692 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:46.112100 (+ 408us) client_negotiation.cc:769] Sending connection context
0807 00:01:46.112313 (+ 213us) client_negotiation.cc:240] Negotiation successful
0807 00:01:46.112716 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":398,"thread_start_us":168,"threads_started":1}
I20260807 00:01:46.115362 746 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40931
I20260807 00:01:46.115703 746 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:46.116307 746 heartbeater.cc:511] Master 127.30.204.126:40931 requested a full tablet report, sending...
I20260807 00:01:46.117520 32419 ts_manager.cc:194] Registered new tserver with Master: 262523c20a6041cc96959e2534ae28c5 (127.30.204.67:36741)
I20260807 00:01:46.118111 32419 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:33251
I20260807 00:01:46.126579 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:01:46.138247 32492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.130531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35960 (local address 127.30.204.126:40931)
0807 00:01:46.130691 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.130695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.130846 (+ 151us) server_negotiation.cc:407] Connection header received
0807 00:01:46.130933 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.130935 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.130984 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.131080 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:46.131936 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.132481 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.133365 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.133599 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.133688 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.135032 (+ 1344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:46.135034 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:46.135036 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:46.135066 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:46.136456 (+ 1390us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.136950 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.136953 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.136954 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.137046 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.137332 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.137334 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.137336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.137591 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:46.137728 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.137999 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.138111 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
I20260807 00:01:46.143826 32419 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35960:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:01:46.147042 32419 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:46.163697 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.158082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39364 (local address 127.30.204.65:38297)
0807 00:01:46.158496 (+ 414us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.158505 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.158531 (+ 26us) server_negotiation.cc:407] Connection header received
0807 00:01:46.158612 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.158618 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.158898 (+ 280us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.159057 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:46.159590 (+ 533us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.160358 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.162011 (+ 1653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.162872 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.162990 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.163171 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.163378 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.163516 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":89,"threads_started":1}
I20260807 00:01:46.163760 769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.156925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:38297 (local address 127.0.0.1:39364)
0807 00:01:46.157719 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:46.157781 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:46.157926 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:46.159058 (+ 1132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:46.159065 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:46.159089 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:46.159270 (+ 181us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.159279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.160521 (+ 1242us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.160525 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.161870 (+ 1345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.161882 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.163011 (+ 1129us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.163015 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.163146 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.163199 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 00:01:46.163329 (+ 130us) client_negotiation.cc:240] Negotiation successful
0807 00:01:46.163562 (+ 233us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":410,"mutex_wait_us":173,"spinlock_wait_cycles":7808,"thread_start_us":112,"threads_started":1}
I20260807 00:01:46.164202 770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.157210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33924 (local address 127.30.204.66:37907)
0807 00:01:46.158670 (+ 1460us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.158683 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.158704 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:01:46.158784 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.158790 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.158992 (+ 202us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.159305 (+ 313us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:46.159584 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.161015 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.162226 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.162996 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.163707 (+ 711us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.163878 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.163934 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.164032 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1365,"thread_start_us":132,"threads_started":1}
I20260807 00:01:46.164230 766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.156932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:37907 (local address 127.0.0.1:33924)
0807 00:01:46.157718 (+ 786us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:46.157781 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:46.157926 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:46.159154 (+ 1228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:46.159170 (+ 16us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:46.159190 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:46.159484 (+ 294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.159490 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.161167 (+ 1677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.161172 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.162039 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.162049 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.163144 (+ 1095us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.163148 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.163264 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.163285 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 00:01:46.163337 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 00:01:46.163562 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":697,"mutex_wait_us":316,"spinlock_wait_cycles":2688,"thread_start_us":129,"threads_started":1}
I20260807 00:01:46.164309 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.158579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39612 (local address 127.30.204.67:36741)
0807 00:01:46.158891 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.158901 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.158921 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:01:46.158990 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.158995 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.159156 (+ 161us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.159304 (+ 148us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:46.159549 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.160801 (+ 1252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.162619 (+ 1818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.163336 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.163838 (+ 502us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.164002 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.164055 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.164172 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":110,"threads_started":1}
I20260807 00:01:46.164577 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.157762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:36741 (local address 127.0.0.1:39612)
0807 00:01:46.158177 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:46.158189 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:46.158269 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:46.159288 (+ 1019us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:46.159291 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:46.159309 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:46.159428 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.159435 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.160959 (+ 1524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.160963 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.162493 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.162503 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.163490 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.163494 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.163594 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.163617 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:01:46.163669 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 00:01:46.163731 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"mutex_wait_us":633,"thread_start_us":102,"threads_started":1}
I20260807 00:01:46.167413 412 tablet_service.cc:1467] Processing CreateTablet for tablet 0435a20ef10041a59f3864378e6aaae3 (DEFAULT_TABLE table=test-table [id=7c6607b3b7144d0b89d6fee8fd3668e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:46.167531 632 tablet_service.cc:1467] Processing CreateTablet for tablet 0435a20ef10041a59f3864378e6aaae3 (DEFAULT_TABLE table=test-table [id=7c6607b3b7144d0b89d6fee8fd3668e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:46.168573 412 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0435a20ef10041a59f3864378e6aaae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:46.168211 32648 tablet_service.cc:1467] Processing CreateTablet for tablet 0435a20ef10041a59f3864378e6aaae3 (DEFAULT_TABLE table=test-table [id=7c6607b3b7144d0b89d6fee8fd3668e4]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:46.168665 632 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0435a20ef10041a59f3864378e6aaae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:46.169255 32648 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0435a20ef10041a59f3864378e6aaae3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:46.175226 775 tablet_bootstrap.cc:492] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486: Bootstrap starting.
I20260807 00:01:46.176857 778 tablet_bootstrap.cc:492] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5: Bootstrap starting.
I20260807 00:01:46.177408 779 tablet_bootstrap.cc:492] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31: Bootstrap starting.
I20260807 00:01:46.178727 775 tablet_bootstrap.cc:654] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:46.179231 778 tablet_bootstrap.cc:654] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:46.179311 779 tablet_bootstrap.cc:654] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:46.180104 779 log.cc:824] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:46.180194 775 log.cc:824] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:46.180500 778 log.cc:824] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:46.182574 775 tablet_bootstrap.cc:492] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486: No bootstrap required, opened a new log
I20260807 00:01:46.182574 779 tablet_bootstrap.cc:492] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31: No bootstrap required, opened a new log
I20260807 00:01:46.182601 778 tablet_bootstrap.cc:492] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5: No bootstrap required, opened a new log
I20260807 00:01:46.182816 775 ts_tablet_manager.cc:1397] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486: Time spent bootstrapping tablet: real 0.008s user 0.002s sys 0.002s
I20260807 00:01:46.182816 779 ts_tablet_manager.cc:1397] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:01:46.183022 778 ts_tablet_manager.cc:1397] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:01:46.186458 778 raft_consensus.cc:348] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.186765 778 raft_consensus.cc:374] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:46.186834 778 raft_consensus.cc:729] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 262523c20a6041cc96959e2534ae28c5, State: Initialized, Role: FOLLOWER
I20260807 00:01:46.187309 778 consensus_queue.cc:260] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.187179 775 raft_consensus.cc:348] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.187520 775 raft_consensus.cc:374] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:46.187595 775 raft_consensus.cc:729] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a11326f5d79c4f24b4ab698f3afcb486, State: Initialized, Role: FOLLOWER
I20260807 00:01:46.187637 779 raft_consensus.cc:348] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.187896 779 raft_consensus.cc:374] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:46.187948 779 raft_consensus.cc:729] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 90e3bd31d16b41c088cc2c4d61668c31, State: Initialized, Role: FOLLOWER
I20260807 00:01:46.188093 775 consensus_queue.cc:260] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.188493 778 ts_tablet_manager.cc:1428] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260807 00:01:46.188483 779 consensus_queue.cc:260] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.188897 746 heartbeater.cc:503] Master 127.30.204.126:40931 was elected leader, sending a full tablet report...
I20260807 00:01:46.189261 775 ts_tablet_manager.cc:1428] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486: Time spent starting tablet: real 0.006s user 0.007s sys 0.001s
I20260807 00:01:46.189466 779 ts_tablet_manager.cc:1428] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 00:01:46.189662 517 heartbeater.cc:503] Master 127.30.204.126:40931 was elected leader, sending a full tablet report...
I20260807 00:01:46.190028 32760 heartbeater.cc:503] Master 127.30.204.126:40931 was elected leader, sending a full tablet report...
I20260807 00:01:46.194339 787 raft_consensus.cc:482] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:46.194559 787 raft_consensus.cc:504] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.195844 787 leader_election.cc:288] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 90e3bd31d16b41c088cc2c4d61668c31 (127.30.204.65:38297), 262523c20a6041cc96959e2534ae28c5 (127.30.204.67:36741)
I20260807 00:01:46.199424 521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.196062 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:38297 (local address 127.30.204.66:59109)
0807 00:01:46.196278 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:46.196292 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:46.196372 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:46.196654 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:46.196657 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:46.196676 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:46.196791 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.196797 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.197506 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.197510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.198293 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.198303 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.198961 (+ 658us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.198964 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.199103 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.199128 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:01:46.199202 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 00:01:46.199280 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141}
I20260807 00:01:46.199506 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.196305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:59109 (local address 127.30.204.65:38297)
0807 00:01:46.196437 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.196442 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.196454 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:01:46.196498 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.196500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.196536 (+ 36us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.196640 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:46.196892 (+ 252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.197403 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.198417 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.198845 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.198961 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.199018 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.199246 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.199337 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:01:46.199892 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.196236 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:49231 (local address 127.30.204.67:36741)
0807 00:01:46.196364 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.196368 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.196821 (+ 453us) server_negotiation.cc:407] Connection header received
0807 00:01:46.196923 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.196926 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.196971 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.197032 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:46.197282 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.197766 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.198778 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.199189 (+ 411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.199303 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.199348 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.199694 (+ 346us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.199777 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:01:46.200150 32682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0435a20ef10041a59f3864378e6aaae3" candidate_uuid: "a11326f5d79c4f24b4ab698f3afcb486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e3bd31d16b41c088cc2c4d61668c31" is_pre_election: true
I20260807 00:01:46.200258 790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.196095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:36741 (local address 127.30.204.66:49231)
0807 00:01:46.196724 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:46.196736 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:46.196842 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:46.197048 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:46.197051 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:46.197067 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:46.197181 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.197186 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.197866 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.197869 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.198659 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:46.198669 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.199409 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.199413 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:46.199543 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.199567 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:01:46.200033 (+ 466us) client_negotiation.cc:240] Negotiation successful
0807 00:01:46.200111 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":568,"mutex_wait_us":260,"thread_start_us":127,"threads_started":1}
I20260807 00:01:46.200516 32682 raft_consensus.cc:2454] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a11326f5d79c4f24b4ab698f3afcb486 in term 0.
I20260807 00:01:46.201012 337 leader_election.cc:302] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e3bd31d16b41c088cc2c4d61668c31, a11326f5d79c4f24b4ab698f3afcb486; no voters:
I20260807 00:01:46.201239 667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0435a20ef10041a59f3864378e6aaae3" candidate_uuid: "a11326f5d79c4f24b4ab698f3afcb486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262523c20a6041cc96959e2534ae28c5" is_pre_election: true
I20260807 00:01:46.201295 787 raft_consensus.cc:2781] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:46.201370 787 raft_consensus.cc:482] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:46.201417 787 raft_consensus.cc:3037] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:46.201538 667 raft_consensus.cc:2454] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a11326f5d79c4f24b4ab698f3afcb486 in term 0.
I20260807 00:01:46.202682 787 raft_consensus.cc:504] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.203063 787 leader_election.cc:288] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [CANDIDATE]: Term 1 election: Requested vote from peers 90e3bd31d16b41c088cc2c4d61668c31 (127.30.204.65:38297), 262523c20a6041cc96959e2534ae28c5 (127.30.204.67:36741)
I20260807 00:01:46.203404 32682 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0435a20ef10041a59f3864378e6aaae3" candidate_uuid: "a11326f5d79c4f24b4ab698f3afcb486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "90e3bd31d16b41c088cc2c4d61668c31"
I20260807 00:01:46.203552 667 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0435a20ef10041a59f3864378e6aaae3" candidate_uuid: "a11326f5d79c4f24b4ab698f3afcb486" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "262523c20a6041cc96959e2534ae28c5"
I20260807 00:01:46.203569 32682 raft_consensus.cc:3037] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:46.203694 667 raft_consensus.cc:3037] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:46.204685 32682 raft_consensus.cc:2454] T 0435a20ef10041a59f3864378e6aaae3 P 90e3bd31d16b41c088cc2c4d61668c31 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a11326f5d79c4f24b4ab698f3afcb486 in term 1.
I20260807 00:01:46.204787 667 raft_consensus.cc:2454] T 0435a20ef10041a59f3864378e6aaae3 P 262523c20a6041cc96959e2534ae28c5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a11326f5d79c4f24b4ab698f3afcb486 in term 1.
I20260807 00:01:46.205008 337 leader_election.cc:302] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 90e3bd31d16b41c088cc2c4d61668c31, a11326f5d79c4f24b4ab698f3afcb486; no voters:
I20260807 00:01:46.205194 787 raft_consensus.cc:2781] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:46.205423 787 raft_consensus.cc:686] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [term 1 LEADER]: Becoming Leader. State: Replica: a11326f5d79c4f24b4ab698f3afcb486, State: Running, Role: LEADER
I20260807 00:01:46.205760 787 consensus_queue.cc:237] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } }
I20260807 00:01:46.209481 32419 catalog_manager.cc:5495] T 0435a20ef10041a59f3864378e6aaae3 P a11326f5d79c4f24b4ab698f3afcb486 reported cstate change: term changed from 0 to 1, leader changed from <none> to a11326f5d79c4f24b4ab698f3afcb486 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "a11326f5d79c4f24b4ab698f3afcb486" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "90e3bd31d16b41c088cc2c4d61668c31" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 38297 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a11326f5d79c4f24b4ab698f3afcb486" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 37907 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "262523c20a6041cc96959e2534ae28c5" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 36741 } health_report { overall_health: UNKNOWN } } }
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[32350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060905, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:01:46.234598 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.225285 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39376 (local address 127.30.204.65:38297)
0807 00:01:46.225446 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.225450 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.225606 (+ 156us) server_negotiation.cc:407] Connection header received
0807 00:01:46.225695 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.225697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.225745 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.225807 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:46.226494 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.227040 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.227937 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.228187 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.228290 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.230821 (+ 2531us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:46.230839 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:46.230856 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:46.230889 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:46.232737 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.233262 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.233267 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.233269 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.233375 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.233702 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.233705 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.233706 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.233980 (+ 274us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:46.234144 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.234311 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.234419 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[32350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[32350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060906, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:01:46.264801 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.255441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39378 (local address 127.30.204.65:38297)
0807 00:01:46.255620 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.255626 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.255877 (+ 251us) server_negotiation.cc:407] Connection header received
0807 00:01:46.256043 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.256046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.256093 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.256179 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:46.256922 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.257481 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.258344 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.258569 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.258665 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.261358 (+ 2693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:46.261362 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:46.261364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:46.261398 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:46.262951 (+ 1553us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.263438 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.263442 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.263443 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.263579 (+ 136us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.263858 (+ 279us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.263861 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.263863 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.264053 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:46.264175 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.264492 (+ 317us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.264640 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[32350](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[32350](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060906, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:01:46.296154 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:46.287292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39392 (local address 127.30.204.65:38297)
0807 00:01:46.287425 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:46.287429 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:46.287734 (+ 305us) server_negotiation.cc:407] Connection header received
0807 00:01:46.287849 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:46.287852 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:46.287907 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:46.287969 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:46.288883 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.289446 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.290302 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:46.290529 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:46.290639 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:46.293137 (+ 2498us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:46.293140 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:46.293142 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:46.293177 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:46.294444 (+ 1267us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.294889 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.294893 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.294894 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.294983 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:46.295269 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:46.295272 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:46.295274 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:46.295479 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:46.295608 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:46.295861 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 00:01:46.295986 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
W20260807 00:01:46.297771 32603 tablet_service.cc:2908] Rejecting scan request for tablet 0435a20ef10041a59f3864378e6aaae3: Uninitialized: clean time has not yet been initialized
I20260807 00:01:46.300704 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 32528
I20260807 00:01:46.316900 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 302
I20260807 00:01:46.334374 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 526
I20260807 00:01:46.357246 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 32373
2026-08-07T00:01:46Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2518 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[821](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[821](info): set up 2 sockets
Aug 07 00:01:46 dist-test-slave-pww0 krb5kdc[821](info): commencing operation
krb5kdc: starting...
W20260807 00:01:47.534229 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.150s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:01:47 dist-test-slave-pww0 krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:01:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:01:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:01:47.671852 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37265
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:37265
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:47.789224 841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:47.789563 841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:47.789629 841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:47.793840 841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:01:47.793918 841 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 00:01:47.793943 841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:47.793967 841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:47.793987 841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:01:47.794014 841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:01:47.799690 841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:37265
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37265
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:47.800907 841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:47.802013 841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:47.810880 855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:47.810894 850 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:47.811041 841 server_base.cc:1034] running on GCE node
W20260807 00:01:47.811249 851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:47.811866 841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:47.813095 841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:47.814291 841 hybrid_clock.cc:648] HybridClock initialized: now 1786060907814261 us; error 44 us; skew 500 ppm
Aug 07 00:01:47 dist-test-slave-pww0 krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:47.821313 841 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:01:47.822391 841 webserver.cc:460] Webserver started at http://127.30.204.126:36365/ using document root <none> and password file <none>
I20260807 00:01:47.822985 841 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:47.823053 841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:47.823288 841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:47.825667 841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "396de37ec63d477886fc2c07ffbf902e"
format_stamp: "Formatted at 2026-08-07 00:01:47 on dist-test-slave-pww0"
server_key: "e7f38471653b9d2b0e1da7dc22b81d83"
server_key_iv: "e7df0aa2f2da457fe6dab3877c489ad8"
server_key_version: "encryptionkey@0"
I20260807 00:01:47.826292 841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "396de37ec63d477886fc2c07ffbf902e"
format_stamp: "Formatted at 2026-08-07 00:01:47 on dist-test-slave-pww0"
server_key: "e7f38471653b9d2b0e1da7dc22b81d83"
server_key_iv: "e7df0aa2f2da457fe6dab3877c489ad8"
server_key_version: "encryptionkey@0"
I20260807 00:01:47.830663 841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 00:01:47.833441 867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:47.834873 841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:47.835021 841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "396de37ec63d477886fc2c07ffbf902e"
format_stamp: "Formatted at 2026-08-07 00:01:47 on dist-test-slave-pww0"
server_key: "e7f38471653b9d2b0e1da7dc22b81d83"
server_key_iv: "e7df0aa2f2da457fe6dab3877c489ad8"
server_key_version: "encryptionkey@0"
I20260807 00:01:47.835137 841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:47.855100 841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:47.858721 841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:47.858932 841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:47.867352 841 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:37265
I20260807 00:01:47.867415 958 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:37265 every 8 connection(s)
I20260807 00:01:47.868449 841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:01:47.870311 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 841
I20260807 00:01:47.870440 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:01:47.870733 31537 external_mini_cluster.cc:1428] Setting key cdd9ae5b4f11b70124378df6089237a9
I20260807 00:01:47.872138 959 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:47.879096 959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e: Bootstrap starting.
Aug 07 00:01:47 dist-test-slave-pww0 krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060907, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:47.881975 959 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:47.882759 959 log.cc:824] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:47.885082 959 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e: No bootstrap required, opened a new log
I20260807 00:01:47.886831 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:47.872300 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35332 (local address 127.30.204.126:37265)
0807 00:01:47.872758 (+ 458us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:47.872775 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:47.872815 (+ 40us) server_negotiation.cc:407] Connection header received
0807 00:01:47.873494 (+ 679us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:47.873513 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:47.873859 (+ 346us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:47.874164 (+ 305us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:47.875096 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:47.876195 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:47.877358 (+ 1163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:47.878000 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:47.878141 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:47.881225 (+ 3084us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:47.881230 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:47.881246 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:47.881284 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:47.883815 (+ 2531us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:47.884331 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:47.884338 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:47.884343 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:47.884471 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:47.884757 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:47.884760 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:47.884762 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:47.885198 (+ 436us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:47.885379 (+ 181us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:47.885712 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 00:01:47.886025 (+ 313us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":301,"thread_start_us":129,"threads_started":1}
I20260807 00:01:47.888725 959 raft_consensus.cc:348] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396de37ec63d477886fc2c07ffbf902e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37265 } }
I20260807 00:01:47.889031 959 raft_consensus.cc:374] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:47.889096 959 raft_consensus.cc:729] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 396de37ec63d477886fc2c07ffbf902e, State: Initialized, Role: FOLLOWER
I20260807 00:01:47.889554 959 consensus_queue.cc:260] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396de37ec63d477886fc2c07ffbf902e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37265 } }
I20260807 00:01:47.889709 959 raft_consensus.cc:388] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:01:47.889779 959 raft_consensus.cc:482] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:01:47.889882 959 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:47.890905 959 raft_consensus.cc:504] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396de37ec63d477886fc2c07ffbf902e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37265 } }
I20260807 00:01:47.891258 959 leader_election.cc:302] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 396de37ec63d477886fc2c07ffbf902e; no voters:
I20260807 00:01:47.891615 959 leader_election.cc:288] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:01:47.891749 970 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:47.891993 970 raft_consensus.cc:686] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [term 1 LEADER]: Becoming Leader. State: Replica: 396de37ec63d477886fc2c07ffbf902e, State: Running, Role: LEADER
I20260807 00:01:47.892323 970 consensus_queue.cc:237] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396de37ec63d477886fc2c07ffbf902e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37265 } }
I20260807 00:01:47.892949 959 sys_catalog.cc:564] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:01:47.894523 973 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "396de37ec63d477886fc2c07ffbf902e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396de37ec63d477886fc2c07ffbf902e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37265 } } }
I20260807 00:01:47.894580 974 sys_catalog.cc:455] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 396de37ec63d477886fc2c07ffbf902e. Latest consensus state: current_term: 1 leader_uuid: "396de37ec63d477886fc2c07ffbf902e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "396de37ec63d477886fc2c07ffbf902e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37265 } } }
I20260807 00:01:47.894677 973 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:47.894692 974 sys_catalog.cc:458] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:47.895115 993 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:01:47.898595 993 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:01:47.904594 993 catalog_manager.cc:1347] Generated new cluster ID: af834deb72d44536879e539864a594c5
I20260807 00:01:47.904688 993 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:01:47.925791 993 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:01:47.926810 993 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:01:47.939980 993 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e: Generated new TSK 0
I20260807 00:01:47.940910 993 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:01:47.983364 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37265
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:48.103785 1010 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:48.104084 1010 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:48.104151 1010 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:48.108273 1010 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:48.108353 1010 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:48.108409 1010 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:01:48.108469 1010 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:01:48.114176 1010 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:37265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1010
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:48.115347 1010 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:48.116479 1010 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:48.126843 1023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:48.126847 1022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:48.126843 1025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:48.127352 1010 server_base.cc:1034] running on GCE node
I20260807 00:01:48.127811 1010 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:48.128495 1010 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:48.129703 1010 hybrid_clock.cc:648] HybridClock initialized: now 1786060908129669 us; error 49 us; skew 500 ppm
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:48.132835 1010 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:01:48.133837 1010 webserver.cc:460] Webserver started at http://127.30.204.65:35381/ using document root <none> and password file <none>
I20260807 00:01:48.134410 1010 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:48.134475 1010 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:48.134653 1010 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:48.136845 1010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "513b5c5526b543dd9824b0f93a354001"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "a095176129292d570001601a7b9d3c6e"
server_key_iv: "7153a1c1d0be8133e97822919b750751"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.137413 1010 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "513b5c5526b543dd9824b0f93a354001"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "a095176129292d570001601a7b9d3c6e"
server_key_iv: "7153a1c1d0be8133e97822919b750751"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.141469 1010 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:01:48.144420 1033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:48.145826 1010 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:48.145953 1010 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "513b5c5526b543dd9824b0f93a354001"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "a095176129292d570001601a7b9d3c6e"
server_key_iv: "7153a1c1d0be8133e97822919b750751"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.146073 1010 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:48.166987 1010 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:48.170581 1010 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:48.170778 1010 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:48.172483 1010 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:48.172540 1010 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:48.172595 1010 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:48.172626 1010 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:48.183851 1010 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:44291
I20260807 00:01:48.183921 1237 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:44291 every 8 connection(s)
I20260807 00:01:48.184947 1010 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:01:48.186829 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.173329 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:33621 (local address 127.30.204.126:37265)
0807 00:01:48.173503 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.173509 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.174450 (+ 941us) server_negotiation.cc:407] Connection header received
0807 00:01:48.175620 (+ 1170us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.175624 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.175696 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.175808 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:48.177365 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.177955 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.179214 (+ 1259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.179511 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.179624 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.182693 (+ 3069us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:48.182698 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:48.182701 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:48.182736 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:48.184399 (+ 1663us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.185019 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.185023 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.185025 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.185158 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.185582 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.185586 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.185588 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.185863 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:48.185964 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.186494 (+ 530us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.186653 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:01:48.187717 1050 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.173763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37265 (local address 127.30.204.65:33621)
0807 00:01:48.174263 (+ 500us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.174319 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.175334 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.176017 (+ 683us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.176029 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.176466 (+ 437us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:48.177174 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.177198 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.178096 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.178099 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.179070 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.179081 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.179704 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.179708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.179898 (+ 190us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.180513 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:48.180539 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:48.182457 (+ 1918us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:48.184527 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.184533 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.184548 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.184891 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.185261 (+ 370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.185264 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.185266 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.185457 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.185977 (+ 520us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:48.185984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:48.186248 (+ 264us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.186426 (+ 178us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.186787 (+ 361us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":116,"threads_started":1}
I20260807 00:01:48.189181 1240 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37265
I20260807 00:01:48.189349 1240 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:48.189890 1240 heartbeater.cc:511] Master 127.30.204.126:37265 requested a full tablet report, sending...
I20260807 00:01:48.191778 890 ts_manager.cc:194] Registered new tserver with Master: 513b5c5526b543dd9824b0f93a354001 (127.30.204.65:44291)
I20260807 00:01:48.192373 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1010
I20260807 00:01:48.192484 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:01:48.192863 31537 external_mini_cluster.cc:1428] Setting key 8abf3d4b0303077d2a2b4a3051b71644
I20260807 00:01:48.193240 890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:33621
I20260807 00:01:48.199030 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.191478 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:39829 (local address 127.30.204.126:37265)
0807 00:01:48.191605 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.191610 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.191786 (+ 176us) server_negotiation.cc:407] Connection header received
0807 00:01:48.191913 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.191916 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.191966 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.192038 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:48.192639 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.193345 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.194395 (+ 1050us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.194743 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.194831 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.196075 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:48.196079 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:48.196081 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:48.196107 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:48.197420 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.197838 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.197842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.197844 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.197930 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.198191 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.198194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.198196 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.198387 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:48.198478 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.198739 (+ 261us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.198866 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:01:48.199038 1254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.191359 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37265 (local address 127.30.204.65:39829)
0807 00:01:48.191703 (+ 344us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.191719 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.191826 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.192078 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.192081 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.192314 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:48.192522 (+ 208us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.192532 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.193501 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.193504 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.194285 (+ 781us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.194297 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.194861 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.194865 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.195019 (+ 154us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.195567 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:48.195582 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:48.195932 (+ 350us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:48.197530 (+ 1598us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.197534 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.197536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.197744 (+ 208us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.198015 (+ 271us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.198018 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.198020 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.198106 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.198479 (+ 373us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:48.198481 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:48.198576 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.198672 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.198831 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":1,"thread_start_us":156,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:01:48.223841 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37265
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:48.340306 1257 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:48.340612 1257 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:48.340682 1257 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:48.344781 1257 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:48.344848 1257 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:48.344905 1257 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:01:48.344959 1257 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:01:48.350602 1257 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:37265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1257
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:48.351898 1257 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:48.352988 1257 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:48.362871 1265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:48.362883 1266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:48.362946 1257 server_base.cc:1034] running on GCE node
W20260807 00:01:48.362883 1268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:48.363646 1257 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:48.364358 1257 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:48.365532 1257 hybrid_clock.cc:648] HybridClock initialized: now 1786060908365510 us; error 41 us; skew 500 ppm
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:48.368566 1257 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:01:48.369547 1257 webserver.cc:460] Webserver started at http://127.30.204.66:43597/ using document root <none> and password file <none>
I20260807 00:01:48.370136 1257 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:48.370203 1257 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:48.370420 1257 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:48.372613 1257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "907f199d94614b0e96a7435ec0dcf4ac"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "60719040ab8ab5ac2a64417c99f1be40"
server_key_iv: "5a603379bd3970bd637f93a4bdd50c9b"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.373189 1257 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "907f199d94614b0e96a7435ec0dcf4ac"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "60719040ab8ab5ac2a64417c99f1be40"
server_key_iv: "5a603379bd3970bd637f93a4bdd50c9b"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.377218 1257 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:01:48.379885 1276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:48.381184 1257 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:01:48.381304 1257 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "907f199d94614b0e96a7435ec0dcf4ac"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "60719040ab8ab5ac2a64417c99f1be40"
server_key_iv: "5a603379bd3970bd637f93a4bdd50c9b"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.381423 1257 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:48.398300 1257 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:48.401543 1257 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:48.401738 1257 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:48.403257 1257 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:48.403311 1257 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:48.403368 1257 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:48.403395 1257 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:48.413661 1257 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:41645
I20260807 00:01:48.413704 1454 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:41645 every 8 connection(s)
I20260807 00:01:48.414767 1257 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:01:48.417239 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.404140 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:56219 (local address 127.30.204.126:37265)
0807 00:01:48.404291 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.404296 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.405199 (+ 903us) server_negotiation.cc:407] Connection header received
0807 00:01:48.406285 (+ 1086us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.406289 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.406356 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.406409 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:48.408169 (+ 1760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.408816 (+ 647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.410001 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.410229 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.410338 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.413488 (+ 3150us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:48.413492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:48.413495 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:48.413528 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:48.415047 (+ 1519us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.415701 (+ 654us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.415704 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.415706 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.415790 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.416126 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.416129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.416130 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.416321 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:48.416409 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.416930 (+ 521us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.417072 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:01:48.418015 1291 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.404540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37265 (local address 127.30.204.66:56219)
0807 00:01:48.405019 (+ 479us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.405081 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.406015 (+ 934us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.406845 (+ 830us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.406853 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.407318 (+ 465us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:48.407983 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.408005 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.408943 (+ 938us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.408947 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.409886 (+ 939us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.409897 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.410412 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.410416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.410586 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.411763 (+ 1177us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:48.411779 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:48.413281 (+ 1502us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:48.415179 (+ 1898us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.415186 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.415203 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.415598 (+ 395us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.415879 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.415882 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.415885 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.416027 (+ 142us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.416425 (+ 398us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:48.416431 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:48.416688 (+ 257us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.416849 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.417179 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":118,"threads_started":1}
I20260807 00:01:48.419374 1456 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37265
I20260807 00:01:48.419577 1456 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:48.420102 1456 heartbeater.cc:511] Master 127.30.204.126:37265 requested a full tablet report, sending...
I20260807 00:01:48.421245 890 ts_manager.cc:194] Registered new tserver with Master: 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66:41645)
I20260807 00:01:48.421833 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1257
I20260807 00:01:48.421944 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:01:48.422044 890 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:56219
I20260807 00:01:48.422281 31537 external_mini_cluster.cc:1428] Setting key 4a5bba6a81a09f86004e6b56b3db946a
I20260807 00:01:48.429662 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.421843 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:52267 (local address 127.30.204.126:37265)
0807 00:01:48.421962 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.421967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.422307 (+ 340us) server_negotiation.cc:407] Connection header received
0807 00:01:48.422463 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.422467 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.422530 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.422623 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:48.423222 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.423949 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.424805 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.425039 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.425139 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.426482 (+ 1343us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:48.426485 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:48.426487 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:48.426513 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:48.427985 (+ 1472us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.428416 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.428420 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.428422 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.428504 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.428768 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.428771 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.428773 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.429002 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:48.429094 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.429376 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.429492 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:01:48.429668 1466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.421765 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37265 (local address 127.30.204.66:52267)
0807 00:01:48.422209 (+ 444us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.422224 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.422329 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.422676 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.422679 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.422908 (+ 229us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:48.423069 (+ 161us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.423078 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.424095 (+ 1017us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.424099 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.424677 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.424687 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.425152 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.425157 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.425263 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.425821 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:48.425840 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:48.426308 (+ 468us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:48.428094 (+ 1786us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.428098 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.428100 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.428320 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.428588 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.428591 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.428593 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.428683 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.429111 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:48.429114 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:48.429218 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.429305 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.429470 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":130,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:01:48.451663 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37265
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45923
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:48.572855 1473 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:48.573170 1473 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:48.573216 1473 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:48.577404 1473 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:48.577491 1473 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:48.577531 1473 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:01:48.577564 1473 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:01:48.583357 1473 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45923
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:37265
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1473
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:48.584645 1473 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:48.585788 1473 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:48.596798 1483 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:48.596813 1479 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:48.596866 1473 server_base.cc:1034] running on GCE node
W20260807 00:01:48.596771 1481 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:48.597462 1473 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:48.598194 1473 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:48.599377 1473 hybrid_clock.cc:648] HybridClock initialized: now 1786060908599346 us; error 47 us; skew 500 ppm
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:48.602560 1473 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:01:48.603586 1473 webserver.cc:460] Webserver started at http://127.30.204.67:40445/ using document root <none> and password file <none>
I20260807 00:01:48.604175 1473 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:48.604238 1473 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:48.604418 1473 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:48.606596 1473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "35b357b133444a8e9822da7932ff4c02"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "031ae4dbf7cacc2bb9b2ff2b500affcc"
server_key_iv: "8433f8d7ec6d23b9a09352bb844ffe4b"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.607237 1473 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "35b357b133444a8e9822da7932ff4c02"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "031ae4dbf7cacc2bb9b2ff2b500affcc"
server_key_iv: "8433f8d7ec6d23b9a09352bb844ffe4b"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.611724 1473 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:01:48.614600 1495 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:48.616086 1473 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:48.616235 1473 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "35b357b133444a8e9822da7932ff4c02"
format_stamp: "Formatted at 2026-08-07 00:01:48 on dist-test-slave-pww0"
server_key: "031ae4dbf7cacc2bb9b2ff2b500affcc"
server_key_iv: "8433f8d7ec6d23b9a09352bb844ffe4b"
server_key_version: "encryptionkey@0"
I20260807 00:01:48.616353 1473 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:48.640836 1473 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:48.644434 1473 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:48.644658 1473 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:48.646281 1473 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:48.646338 1473 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:48.646414 1473 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:48.646457 1473 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:48.657487 1473 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:40571
I20260807 00:01:48.657516 1675 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:40571 every 8 connection(s)
I20260807 00:01:48.658717 1473 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:01:48.659735 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1473
I20260807 00:01:48.659858 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:01:48.660164 31537 external_mini_cluster.cc:1428] Setting key 2930cef1dde0e6019398d5017a20d5e6
I20260807 00:01:48.661945 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.647315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:56815 (local address 127.30.204.126:37265)
0807 00:01:48.647444 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.647447 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.648665 (+ 1218us) server_negotiation.cc:407] Connection header received
0807 00:01:48.649873 (+ 1208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.649877 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.649946 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.650006 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:48.651948 (+ 1942us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.652572 (+ 624us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.653873 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.654100 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.654208 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.657125 (+ 2917us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:48.657129 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:48.657132 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:48.657170 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:48.659018 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.659926 (+ 908us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.659930 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.659932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.660044 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.660534 (+ 490us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.660572 (+ 38us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.660574 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.660901 (+ 327us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:48.661001 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.661692 (+ 691us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.661822 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:01:48.662943 1505 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.647939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37265 (local address 127.30.204.67:56815)
0807 00:01:48.648493 (+ 554us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.648542 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.649561 (+ 1019us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.650513 (+ 952us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.650521 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.651031 (+ 510us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:48.651770 (+ 739us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.651788 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.652714 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.652718 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.653736 (+ 1018us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.653748 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.654271 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.654275 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.654453 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.655135 (+ 682us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:48.655162 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:48.656904 (+ 1742us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:48.659182 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.659189 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.659206 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.659594 (+ 388us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.660175 (+ 581us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.660178 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.660179 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.660386 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.661061 (+ 675us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:48.661068 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:48.661410 (+ 342us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.661613 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.661991 (+ 378us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":153,"threads_started":1}
I20260807 00:01:48.664145 1676 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37265
I20260807 00:01:48.664288 1676 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:48.664837 1676 heartbeater.cc:511] Master 127.30.204.126:37265 requested a full tablet report, sending...
I20260807 00:01:48.665937 891 ts_manager.cc:194] Registered new tserver with Master: 35b357b133444a8e9822da7932ff4c02 (127.30.204.67:40571)
I20260807 00:01:48.666714 891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:56815
I20260807 00:01:48.673718 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.666379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:54209 (local address 127.30.204.126:37265)
0807 00:01:48.666491 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.666495 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.666727 (+ 232us) server_negotiation.cc:407] Connection header received
0807 00:01:48.666892 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.666894 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.666939 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.667008 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:48.667515 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.668033 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.669091 (+ 1058us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.669321 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.669414 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.670593 (+ 1179us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:48.670596 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:48.670598 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:48.670624 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:48.671972 (+ 1348us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.672391 (+ 419us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.672394 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.672395 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.672486 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.672861 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.672864 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.672865 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.673053 (+ 188us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:48.673153 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.673441 (+ 288us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.673548 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:01:48.673735 1685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.666295 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37265 (local address 127.30.204.67:54209)
0807 00:01:48.666649 (+ 354us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.666664 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.666772 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.667038 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.667041 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.667237 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:48.667368 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.667376 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.668165 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.668169 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.668946 (+ 777us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.668956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.669459 (+ 503us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.669462 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.669541 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.670017 (+ 476us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:48.670028 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:48.670373 (+ 345us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:48.672078 (+ 1705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.672081 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.672084 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.672279 (+ 195us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.672615 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:48.672620 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:48.672623 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:48.672754 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:48.673185 (+ 431us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:48.673188 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:48.673278 (+ 90us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.673370 (+ 92us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.673536 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":271,"thread_start_us":122,"threads_started":1}
I20260807 00:01:48.674494 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:01:48 dist-test-slave-pww0 krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:48.702754 963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.693951 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35344 (local address 127.30.204.126:37265)
0807 00:01:48.694129 (+ 178us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.694132 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.694146 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:01:48.694193 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.694197 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.694251 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.694326 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:48.695134 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.695791 (+ 657us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.696678 (+ 887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.696971 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.697071 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.699659 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:48.699662 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:48.699664 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:48.699704 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:48.700993 (+ 1289us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.701457 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.701460 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.701462 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.701550 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:48.701880 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:48.701882 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:48.701884 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:48.702071 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:48.702180 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.702454 (+ 274us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.702600 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:01:48.706465 891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:01:48.709748 891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:48.726548 1703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.719419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44291 (local address 127.0.0.1:33140)
0807 00:01:48.720342 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.720405 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.720545 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.721372 (+ 827us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.721379 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.721402 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:48.721613 (+ 211us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.721621 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.723117 (+ 1496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.723120 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.724786 (+ 1666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.724792 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.725898 (+ 1106us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.725903 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.726031 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.726096 (+ 65us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.726231 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.726376 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":746,"mutex_wait_us":107,"thread_start_us":182,"threads_started":1}
I20260807 00:01:48.727106 1700 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.719213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40571 (local address 127.0.0.1:40688)
0807 00:01:48.720661 (+ 1448us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.720677 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.720763 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.723001 (+ 2238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.723005 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.723023 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:48.723155 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.723162 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.724440 (+ 1278us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.724444 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.725415 (+ 971us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.725427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.726708 (+ 1281us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.726712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.726816 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.726841 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.726904 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.726976 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":762,"mutex_wait_us":3,"thread_start_us":108,"threads_started":1}
I20260807 00:01:48.727260 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.720188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33140 (local address 127.30.204.65:44291)
0807 00:01:48.720942 (+ 754us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.720949 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.720974 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:01:48.721049 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.721053 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.721226 (+ 173us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.721360 (+ 134us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:48.721755 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.722988 (+ 1233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.724923 (+ 1935us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.725754 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.725844 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.725969 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.726997 (+ 1028us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.727131 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":550,"thread_start_us":95,"threads_started":1}
I20260807 00:01:48.727659 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.719420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40688 (local address 127.30.204.67:40571)
0807 00:01:48.721245 (+ 1825us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.721255 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.721276 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:01:48.721346 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.721351 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.722672 (+ 1321us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.722837 (+ 165us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:48.723294 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.724325 (+ 1031us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.725595 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.726364 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.726465 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.726641 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.727364 (+ 723us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.727506 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1724,"thread_start_us":127,"threads_started":1}
I20260807 00:01:48.728165 1705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.719659 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:41645 (local address 127.0.0.1:57620)
0807 00:01:48.720891 (+ 1232us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.720901 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.720987 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.721528 (+ 541us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.721531 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.721548 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:48.721635 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.721641 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.723088 (+ 1447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.723092 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.724777 (+ 1685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.724791 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.727733 (+ 2942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.727737 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.727845 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.727871 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.727938 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.728018 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1180,"mutex_wait_us":138,"thread_start_us":97,"threads_started":1}
I20260807 00:01:48.728421 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.720168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57620 (local address 127.30.204.66:41645)
0807 00:01:48.720739 (+ 571us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.720751 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.720986 (+ 235us) server_negotiation.cc:407] Connection header received
0807 00:01:48.721092 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.721098 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.721301 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.721445 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:48.721754 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.722934 (+ 1180us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.726036 (+ 3102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.727325 (+ 1289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.727438 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.727665 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.728007 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.728120 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":126,"threads_started":1}
I20260807 00:01:48.730280 1575 tablet_service.cc:1467] Processing CreateTablet for tablet 565322626bd44c86a4e772cb0e41779a (DEFAULT_TABLE table=test-table [id=6c43dc5ee96f4b7284743d235d782cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:48.730252 1127 tablet_service.cc:1467] Processing CreateTablet for tablet 565322626bd44c86a4e772cb0e41779a (DEFAULT_TABLE table=test-table [id=6c43dc5ee96f4b7284743d235d782cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:48.731520 1127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565322626bd44c86a4e772cb0e41779a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:48.731519 1575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565322626bd44c86a4e772cb0e41779a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:48.738479 1709 tablet_bootstrap.cc:492] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Bootstrap starting.
I20260807 00:01:48.738147 1353 tablet_service.cc:1467] Processing CreateTablet for tablet 565322626bd44c86a4e772cb0e41779a (DEFAULT_TABLE table=test-table [id=6c43dc5ee96f4b7284743d235d782cd9]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:48.739432 1353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 565322626bd44c86a4e772cb0e41779a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:48.739431 1712 tablet_bootstrap.cc:492] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Bootstrap starting.
I20260807 00:01:48.741792 1712 tablet_bootstrap.cc:654] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:48.742306 1709 tablet_bootstrap.cc:654] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:48.742770 1712 log.cc:824] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:48.743165 1709 log.cc:824] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:48.744917 1712 tablet_bootstrap.cc:492] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: No bootstrap required, opened a new log
I20260807 00:01:48.745131 1712 ts_tablet_manager.cc:1397] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 00:01:48.745325 1709 tablet_bootstrap.cc:492] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: No bootstrap required, opened a new log
I20260807 00:01:48.745533 1709 ts_tablet_manager.cc:1397] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Time spent bootstrapping tablet: real 0.007s user 0.003s sys 0.002s
I20260807 00:01:48.747524 1716 tablet_bootstrap.cc:492] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Bootstrap starting.
I20260807 00:01:48.749577 1709 raft_consensus.cc:348] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.749851 1709 raft_consensus.cc:374] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:48.749907 1709 raft_consensus.cc:729] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b357b133444a8e9822da7932ff4c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:48.749737 1712 raft_consensus.cc:348] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.750001 1712 raft_consensus.cc:374] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:48.750067 1712 raft_consensus.cc:729] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b5c5526b543dd9824b0f93a354001, State: Initialized, Role: FOLLOWER
I20260807 00:01:48.750480 1709 consensus_queue.cc:260] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.750545 1712 consensus_queue.cc:260] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.750662 1716 tablet_bootstrap.cc:654] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:48.751236 1676 heartbeater.cc:503] Master 127.30.204.126:37265 was elected leader, sending a full tablet report...
I20260807 00:01:48.751639 1709 ts_tablet_manager.cc:1428] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 00:01:48.751763 1716 log.cc:824] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:48.752436 1240 heartbeater.cc:503] Master 127.30.204.126:37265 was elected leader, sending a full tablet report...
I20260807 00:01:48.753062 1712 ts_tablet_manager.cc:1428] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Time spent starting tablet: real 0.008s user 0.005s sys 0.002s
I20260807 00:01:48.754315 1716 tablet_bootstrap.cc:492] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: No bootstrap required, opened a new log
I20260807 00:01:48.754545 1716 ts_tablet_manager.cc:1397] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:01:48.758543 1716 raft_consensus.cc:348] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.758842 1716 raft_consensus.cc:374] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:48.758893 1716 raft_consensus.cc:729] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907f199d94614b0e96a7435ec0dcf4ac, State: Initialized, Role: FOLLOWER
I20260807 00:01:48.759323 1716 consensus_queue.cc:260] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.760119 1456 heartbeater.cc:503] Master 127.30.204.126:37265 was elected leader, sending a full tablet report...
I20260807 00:01:48.760382 1716 ts_tablet_manager.cc:1428] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
W20260807 00:01:48.909915 1677 tablet.cc:2369] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:01:48.916072 1458 tablet.cc:2369] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:01:48.927080 1722 raft_consensus.cc:482] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:48.927335 1722 raft_consensus.cc:504] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.928589 1722 leader_election.cc:288] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 35b357b133444a8e9822da7932ff4c02 (127.30.204.67:40571), 513b5c5526b543dd9824b0f93a354001 (127.30.204.65:44291)
I20260807 00:01:48.932929 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.928958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:45053 (local address 127.30.204.67:40571)
0807 00:01:48.929096 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.929101 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.929344 (+ 243us) server_negotiation.cc:407] Connection header received
0807 00:01:48.929497 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.929500 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.929550 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.929624 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:48.930002 (+ 378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.930554 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.931869 (+ 1315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.932317 (+ 448us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.932454 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.932515 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.932735 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.932817 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:01:48.932960 1724 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.928838 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40571 (local address 127.30.204.66:45053)
0807 00:01:48.929218 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.929232 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.929378 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.929631 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.929633 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.929660 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:48.929879 (+ 219us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.929885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.930645 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.930648 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.931704 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.931716 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.932454 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.932458 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.932566 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.932613 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.932686 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.932769 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":48,"spinlock_wait_cycles":22912,"thread_start_us":103,"threads_started":1}
I20260807 00:01:48.933465 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.929012 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:57501 (local address 127.30.204.65:44291)
0807 00:01:48.929162 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.929167 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.929393 (+ 226us) server_negotiation.cc:407] Connection header received
0807 00:01:48.929464 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.929468 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.929521 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.929616 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:48.930687 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.931165 (+ 478us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.932295 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.932738 (+ 443us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.932857 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.932907 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.933252 (+ 345us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.933349 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:01:48.933646 1609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565322626bd44c86a4e772cb0e41779a" candidate_uuid: "907f199d94614b0e96a7435ec0dcf4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b357b133444a8e9822da7932ff4c02" is_pre_election: true
I20260807 00:01:48.933835 1723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.928927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44291 (local address 127.30.204.66:57501)
0807 00:01:48.929219 (+ 292us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:48.929232 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:48.929378 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:48.929629 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:48.929632 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:48.929652 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:48.929876 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.929884 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.931275 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.931279 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.932169 (+ 890us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:48.932180 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.932967 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.932971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:48.933084 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.933109 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:01:48.933617 (+ 508us) client_negotiation.cc:240] Negotiation successful
0807 00:01:48.933701 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":227,"mutex_wait_us":28,"spinlock_wait_cycles":31616,"thread_start_us":65,"threads_started":1}
I20260807 00:01:48.934008 1609 raft_consensus.cc:2454] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 907f199d94614b0e96a7435ec0dcf4ac in term 0.
I20260807 00:01:48.934480 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565322626bd44c86a4e772cb0e41779a" candidate_uuid: "907f199d94614b0e96a7435ec0dcf4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b5c5526b543dd9824b0f93a354001" is_pre_election: true
I20260807 00:01:48.934618 1280 leader_election.cc:302] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b357b133444a8e9822da7932ff4c02, 907f199d94614b0e96a7435ec0dcf4ac; no voters:
I20260807 00:01:48.934775 1153 raft_consensus.cc:2454] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 907f199d94614b0e96a7435ec0dcf4ac in term 0.
I20260807 00:01:48.934927 1722 raft_consensus.cc:2781] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:48.935035 1722 raft_consensus.cc:482] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:48.935106 1722 raft_consensus.cc:3037] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:48.936267 1722 raft_consensus.cc:504] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
W20260807 00:01:48.936365 1246 tablet.cc:2369] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:01:48.936635 1722 leader_election.cc:288] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [CANDIDATE]: Term 1 election: Requested vote from peers 35b357b133444a8e9822da7932ff4c02 (127.30.204.67:40571), 513b5c5526b543dd9824b0f93a354001 (127.30.204.65:44291)
I20260807 00:01:48.937022 1609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565322626bd44c86a4e772cb0e41779a" candidate_uuid: "907f199d94614b0e96a7435ec0dcf4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b357b133444a8e9822da7932ff4c02"
I20260807 00:01:48.937052 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "565322626bd44c86a4e772cb0e41779a" candidate_uuid: "907f199d94614b0e96a7435ec0dcf4ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b5c5526b543dd9824b0f93a354001"
I20260807 00:01:48.937172 1153 raft_consensus.cc:3037] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:48.937173 1609 raft_consensus.cc:3037] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:48.938621 1153 raft_consensus.cc:2454] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 907f199d94614b0e96a7435ec0dcf4ac in term 1.
I20260807 00:01:48.938967 1609 raft_consensus.cc:2454] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 907f199d94614b0e96a7435ec0dcf4ac in term 1.
I20260807 00:01:48.939029 1277 leader_election.cc:302] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513b5c5526b543dd9824b0f93a354001, 907f199d94614b0e96a7435ec0dcf4ac; no voters:
I20260807 00:01:48.939230 1722 raft_consensus.cc:2781] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:48.939427 1722 raft_consensus.cc:686] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 1 LEADER]: Becoming Leader. State: Replica: 907f199d94614b0e96a7435ec0dcf4ac, State: Running, Role: LEADER
I20260807 00:01:48.939813 1722 consensus_queue.cc:237] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } }
I20260807 00:01:48.943233 890 catalog_manager.cc:5495] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66). New cstate: current_term: 1 leader_uuid: "907f199d94614b0e96a7435ec0dcf4ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } health_report { overall_health: HEALTHY } } }
I20260807 00:01:48.969779 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:48.965794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57630 (local address 127.30.204.66:41645)
0807 00:01:48.965983 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:48.965987 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:48.966005 (+ 18us) server_negotiation.cc:407] Connection header received
0807 00:01:48.966216 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:48.966218 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:48.966279 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:48.966367 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:48.966612 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.967116 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.968075 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:48.968316 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:48.968411 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:48.968732 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:48.969202 (+ 470us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:48.969314 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:48.969527 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 00:01:48.969606 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 00:01:48.976363 1153 raft_consensus.cc:1261] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Refusing update from remote peer 907f199d94614b0e96a7435ec0dcf4ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:48.976363 1609 raft_consensus.cc:1261] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Refusing update from remote peer 907f199d94614b0e96a7435ec0dcf4ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:48.977145 1726 consensus_queue.cc:1035] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:48.977370 1722 consensus_queue.cc:1035] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:48.984697 1735 mvcc.cc:205] Tried to move back new op lower bound from 7315705483159289856 to 7315705483020066816. Current Snapshot: MvccSnapshot[applied={T|T < 7315705483159289856}]
I20260807 00:01:48.985466 1736 mvcc.cc:205] Tried to move back new op lower bound from 7315705483159289856 to 7315705483020066816. Current Snapshot: MvccSnapshot[applied={T|T < 7315705483159289856}]
I20260807 00:01:49.000630 891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:49.000976 891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:49.004138 891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e: Sending DeleteTablet for 3 replicas of tablet 565322626bd44c86a4e772cb0e41779a
I20260807 00:01:49.005153 1127 tablet_service.cc:1514] Processing DeleteTablet for tablet 565322626bd44c86a4e772cb0e41779a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:48 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:33140
I20260807 00:01:49.005154 1353 tablet_service.cc:1514] Processing DeleteTablet for tablet 565322626bd44c86a4e772cb0e41779a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:48 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:57620
I20260807 00:01:49.005167 1575 tablet_service.cc:1514] Processing DeleteTablet for tablet 565322626bd44c86a4e772cb0e41779a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:48 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:40688
I20260807 00:01:49.005820 1748 tablet_replica.cc:330] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: stopping tablet replica
I20260807 00:01:49.006038 1749 tablet_replica.cc:330] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: stopping tablet replica
I20260807 00:01:49.006254 1748 raft_consensus.cc:2229] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:49.006508 1749 raft_consensus.cc:2229] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:01:49.006644 1748 raft_consensus.cc:2258] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:49.007052 1749 raft_consensus.cc:2258] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:49.007104 885 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35344:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:01:49.007635 885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:49.008404 1748 ts_tablet_manager.cc:1905] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:49.008949 1749 ts_tablet_manager.cc:1905] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:49.011387 1748 ts_tablet_manager.cc:1918] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:01:49.011543 1748 log.cc:1196] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/565322626bd44c86a4e772cb0e41779a
I20260807 00:01:49.011849 1750 tablet_replica.cc:330] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: stopping tablet replica
I20260807 00:01:49.011874 1748 ts_tablet_manager.cc:1939] T 565322626bd44c86a4e772cb0e41779a P 513b5c5526b543dd9824b0f93a354001: Deleting consensus metadata
I20260807 00:01:49.012372 1750 raft_consensus.cc:2229] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:49.012799 1750 raft_consensus.cc:2258] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:49.013355 869 catalog_manager.cc:4841] TS 513b5c5526b543dd9824b0f93a354001 (127.30.204.65:44291): tablet 565322626bd44c86a4e772cb0e41779a (table test-table [id=6c43dc5ee96f4b7284743d235d782cd9]) successfully deleted
I20260807 00:01:49.013684 1749 ts_tablet_manager.cc:1918] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:01:49.013794 1749 log.cc:1196] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/565322626bd44c86a4e772cb0e41779a
I20260807 00:01:49.014097 1749 ts_tablet_manager.cc:1939] T 565322626bd44c86a4e772cb0e41779a P 907f199d94614b0e96a7435ec0dcf4ac: Deleting consensus metadata
I20260807 00:01:49.014487 1575 tablet_service.cc:1467] Processing CreateTablet for tablet 621f70b1281c44dfa7bdb944a9758d87 (DEFAULT_TABLE table=test-table [id=8cc086ec045c4d4e9c7b7a2ed3fba96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:49.014487 1127 tablet_service.cc:1467] Processing CreateTablet for tablet 621f70b1281c44dfa7bdb944a9758d87 (DEFAULT_TABLE table=test-table [id=8cc086ec045c4d4e9c7b7a2ed3fba96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:49.014746 1575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621f70b1281c44dfa7bdb944a9758d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:49.014746 1127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621f70b1281c44dfa7bdb944a9758d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:49.016347 1750 ts_tablet_manager.cc:1905] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:49.017149 1709 tablet_bootstrap.cc:492] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: Bootstrap starting.
I20260807 00:01:49.017149 1712 tablet_bootstrap.cc:492] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: Bootstrap starting.
I20260807 00:01:49.018122 1712 tablet_bootstrap.cc:654] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:49.018121 1709 tablet_bootstrap.cc:654] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:49.018898 1750 ts_tablet_manager.cc:1918] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:01:49.019033 1750 log.cc:1196] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/565322626bd44c86a4e772cb0e41779a
I20260807 00:01:49.019413 1750 ts_tablet_manager.cc:1939] T 565322626bd44c86a4e772cb0e41779a P 35b357b133444a8e9822da7932ff4c02: Deleting consensus metadata
I20260807 00:01:49.019737 1353 tablet_service.cc:1467] Processing CreateTablet for tablet 621f70b1281c44dfa7bdb944a9758d87 (DEFAULT_TABLE table=test-table [id=8cc086ec045c4d4e9c7b7a2ed3fba96d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:49.020061 1353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 621f70b1281c44dfa7bdb944a9758d87. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:49.020115 1712 tablet_bootstrap.cc:492] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: No bootstrap required, opened a new log
I20260807 00:01:49.020210 1712 ts_tablet_manager.cc:1397] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 00:01:49.021009 1712 raft_consensus.cc:348] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.021137 1712 raft_consensus.cc:374] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:49.021180 1712 raft_consensus.cc:729] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b5c5526b543dd9824b0f93a354001, State: Initialized, Role: FOLLOWER
I20260807 00:01:49.021310 1712 consensus_queue.cc:260] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.021356 874 catalog_manager.cc:4841] TS 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66:41645): tablet 565322626bd44c86a4e772cb0e41779a (table test-table [id=6c43dc5ee96f4b7284743d235d782cd9]) successfully deleted
I20260807 00:01:49.021587 1712 ts_tablet_manager.cc:1428] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260807 00:01:49.022038 1709 tablet_bootstrap.cc:492] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: No bootstrap required, opened a new log
I20260807 00:01:49.022116 1709 ts_tablet_manager.cc:1397] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 00:01:49.022397 873 catalog_manager.cc:4841] TS 35b357b133444a8e9822da7932ff4c02 (127.30.204.67:40571): tablet 565322626bd44c86a4e772cb0e41779a (table test-table [id=6c43dc5ee96f4b7284743d235d782cd9]) successfully deleted
I20260807 00:01:49.022553 1709 raft_consensus.cc:348] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.022635 1716 tablet_bootstrap.cc:492] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: Bootstrap starting.
I20260807 00:01:49.022653 1709 raft_consensus.cc:374] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:49.022682 1709 raft_consensus.cc:729] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b357b133444a8e9822da7932ff4c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:49.022802 1709 consensus_queue.cc:260] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.023049 1709 ts_tablet_manager.cc:1428] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:49.023635 1716 tablet_bootstrap.cc:654] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:49.025058 1716 tablet_bootstrap.cc:492] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: No bootstrap required, opened a new log
I20260807 00:01:49.025151 1716 ts_tablet_manager.cc:1397] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:49.025619 1716 raft_consensus.cc:348] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.025733 1716 raft_consensus.cc:374] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:49.025763 1716 raft_consensus.cc:729] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907f199d94614b0e96a7435ec0dcf4ac, State: Initialized, Role: FOLLOWER
I20260807 00:01:49.025887 1716 consensus_queue.cc:260] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.026132 1716 ts_tablet_manager.cc:1428] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 00:01:49.232071 1717 raft_consensus.cc:482] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:49.232264 1717 raft_consensus.cc:504] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.233435 1717 leader_election.cc:288] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 513b5c5526b543dd9824b0f93a354001 (127.30.204.65:44291), 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66:41645)
I20260807 00:01:49.237404 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.233853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:40079 (local address 127.30.204.65:44291)
0807 00:01:49.233989 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.233995 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.234129 (+ 134us) server_negotiation.cc:407] Connection header received
0807 00:01:49.234305 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.234308 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.234364 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.234431 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:49.234720 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.235225 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.236431 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.236836 (+ 405us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.236950 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.237010 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.237189 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.237265 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:01:49.237422 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.233830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:44731 (local address 127.30.204.66:41645)
0807 00:01:49.234075 (+ 245us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.234080 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.234112 (+ 32us) server_negotiation.cc:407] Connection header received
0807 00:01:49.234316 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.234319 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.234367 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.234430 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:49.234733 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.235245 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.236429 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.236847 (+ 418us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.236958 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.237016 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.237218 (+ 202us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.237299 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":161}
I20260807 00:01:49.237432 1757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.233755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44291 (local address 127.30.204.67:40079)
0807 00:01:49.234044 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.234057 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.234200 (+ 143us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.234458 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.234461 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.234486 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:49.234615 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.234621 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.235330 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.235334 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.236306 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.236318 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.236951 (+ 633us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.236955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.237054 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.237097 (+ 43us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.237168 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.237254 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":229,"mutex_wait_us":5,"spinlock_wait_cycles":33280,"thread_start_us":76,"threads_started":1}
I20260807 00:01:49.237442 1755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.233739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:41645 (local address 127.30.204.67:44731)
0807 00:01:49.234029 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.234044 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.234202 (+ 158us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.234453 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.234456 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.234485 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:49.234624 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.234629 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.235349 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.235352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.236295 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.236307 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.236958 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.236961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.237097 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.237122 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.237189 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.237271 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"mutex_wait_us":99,"spinlock_wait_cycles":47360,"thread_start_us":78,"threads_started":1}
I20260807 00:01:49.238212 1379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621f70b1281c44dfa7bdb944a9758d87" candidate_uuid: "35b357b133444a8e9822da7932ff4c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907f199d94614b0e96a7435ec0dcf4ac" is_pre_election: true
I20260807 00:01:49.238281 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621f70b1281c44dfa7bdb944a9758d87" candidate_uuid: "35b357b133444a8e9822da7932ff4c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b5c5526b543dd9824b0f93a354001" is_pre_election: true
I20260807 00:01:49.238435 1153 raft_consensus.cc:2454] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b357b133444a8e9822da7932ff4c02 in term 0.
I20260807 00:01:49.238472 1379 raft_consensus.cc:2454] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 35b357b133444a8e9822da7932ff4c02 in term 0.
I20260807 00:01:49.238857 1496 leader_election.cc:302] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b357b133444a8e9822da7932ff4c02, 513b5c5526b543dd9824b0f93a354001; no voters:
I20260807 00:01:49.239156 1717 raft_consensus.cc:2781] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:49.239219 1717 raft_consensus.cc:482] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:49.239260 1717 raft_consensus.cc:3037] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:49.240382 1717 raft_consensus.cc:504] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.240732 1717 leader_election.cc:288] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [CANDIDATE]: Term 1 election: Requested vote from peers 513b5c5526b543dd9824b0f93a354001 (127.30.204.65:44291), 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66:41645)
I20260807 00:01:49.241097 1153 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621f70b1281c44dfa7bdb944a9758d87" candidate_uuid: "35b357b133444a8e9822da7932ff4c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "513b5c5526b543dd9824b0f93a354001"
I20260807 00:01:49.241138 1379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "621f70b1281c44dfa7bdb944a9758d87" candidate_uuid: "35b357b133444a8e9822da7932ff4c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907f199d94614b0e96a7435ec0dcf4ac"
I20260807 00:01:49.241221 1153 raft_consensus.cc:3037] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:49.241262 1379 raft_consensus.cc:3037] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:49.242149 1153 raft_consensus.cc:2454] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b357b133444a8e9822da7932ff4c02 in term 1.
I20260807 00:01:49.242173 1379 raft_consensus.cc:2454] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 35b357b133444a8e9822da7932ff4c02 in term 1.
I20260807 00:01:49.242455 1499 leader_election.cc:302] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b357b133444a8e9822da7932ff4c02, 907f199d94614b0e96a7435ec0dcf4ac; no voters:
I20260807 00:01:49.242643 1717 raft_consensus.cc:2781] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:49.242805 1717 raft_consensus.cc:686] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 1 LEADER]: Becoming Leader. State: Replica: 35b357b133444a8e9822da7932ff4c02, State: Running, Role: LEADER
I20260807 00:01:49.243058 1717 consensus_queue.cc:237] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.245368 885 catalog_manager.cc:5495] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 35b357b133444a8e9822da7932ff4c02 (127.30.204.67). New cstate: current_term: 1 leader_uuid: "35b357b133444a8e9822da7932ff4c02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } health_report { overall_health: HEALTHY } } }
Aug 07 00:01:49 dist-test-slave-pww0 krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060907, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:49.302567 1770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.293432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37265 (local address 127.0.0.1:35352)
0807 00:01:49.293797 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.293812 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.293924 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.294175 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.294178 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.294661 (+ 483us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:49.294764 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.294771 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.295605 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.295608 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.296201 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.296209 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.296721 (+ 512us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.296795 (+ 74us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.296867 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.297457 (+ 590us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:49.297470 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:49.299071 (+ 1601us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:49.300893 (+ 1822us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:49.300906 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:49.300912 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:49.301179 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:49.301510 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:49.301512 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:49.301514 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:49.301611 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:49.302014 (+ 403us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:49.302021 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:49.302144 (+ 123us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.302267 (+ 123us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.302407 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":270,"thread_start_us":124,"threads_started":1}
I20260807 00:01:49.302731 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.293444 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35352 (local address 127.30.204.126:37265)
0807 00:01:49.293761 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.293769 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.293877 (+ 108us) server_negotiation.cc:407] Connection header received
0807 00:01:49.293996 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.293999 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.294042 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.294157 (+ 115us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:49.294881 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.295488 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.296329 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.296576 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.296666 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.299237 (+ 2571us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:49.299242 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:49.299244 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:49.299271 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:49.300764 (+ 1493us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:49.301281 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:49.301284 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:49.301285 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:49.301414 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:49.301692 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:49.301695 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:49.301696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:49.301885 (+ 189us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:49.301958 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.302362 (+ 404us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.302490 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":238,"thread_start_us":124,"threads_started":1}
I20260807 00:01:49.306672 885 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:35352:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 00:01:49.307433 885 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:49.312902 1127 tablet_service.cc:1467] Processing CreateTablet for tablet 41f7bd27d80b45eab548ed29efd971e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17b31b11987241de9eeb6d096e91077d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:01:49.313079 1575 tablet_service.cc:1467] Processing CreateTablet for tablet 41f7bd27d80b45eab548ed29efd971e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17b31b11987241de9eeb6d096e91077d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:01:49.313432 1127 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f7bd27d80b45eab548ed29efd971e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:49.313499 1575 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f7bd27d80b45eab548ed29efd971e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:49.313304 1353 tablet_service.cc:1467] Processing CreateTablet for tablet 41f7bd27d80b45eab548ed29efd971e0 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=17b31b11987241de9eeb6d096e91077d]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:01:49.313858 1353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 41f7bd27d80b45eab548ed29efd971e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:49.316371 1709 tablet_bootstrap.cc:492] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: Bootstrap starting.
I20260807 00:01:49.316677 1712 tablet_bootstrap.cc:492] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: Bootstrap starting.
I20260807 00:01:49.317127 1716 tablet_bootstrap.cc:492] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: Bootstrap starting.
I20260807 00:01:49.317430 1709 tablet_bootstrap.cc:654] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:49.317564 1712 tablet_bootstrap.cc:654] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:49.318039 1716 tablet_bootstrap.cc:654] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:49.318878 1712 tablet_bootstrap.cc:492] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: No bootstrap required, opened a new log
I20260807 00:01:49.318971 1712 ts_tablet_manager.cc:1397] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.001s
I20260807 00:01:49.319113 1709 tablet_bootstrap.cc:492] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: No bootstrap required, opened a new log
I20260807 00:01:49.319186 1709 ts_tablet_manager.cc:1397] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:49.319222 1716 tablet_bootstrap.cc:492] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: No bootstrap required, opened a new log
I20260807 00:01:49.319300 1716 ts_tablet_manager.cc:1397] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:01:49.319442 1712 raft_consensus.cc:348] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.319584 1712 raft_consensus.cc:374] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:49.319625 1712 raft_consensus.cc:729] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 513b5c5526b543dd9824b0f93a354001, State: Initialized, Role: FOLLOWER
I20260807 00:01:49.319674 1709 raft_consensus.cc:348] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.319804 1709 raft_consensus.cc:374] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:49.319733 1716 raft_consensus.cc:348] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.319782 1712 consensus_queue.cc:260] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.319845 1709 raft_consensus.cc:729] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 35b357b133444a8e9822da7932ff4c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:49.319869 1716 raft_consensus.cc:374] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:49.319911 1716 raft_consensus.cc:729] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 907f199d94614b0e96a7435ec0dcf4ac, State: Initialized, Role: FOLLOWER
I20260807 00:01:49.319977 1709 consensus_queue.cc:260] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.320041 1716 consensus_queue.cc:260] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.320061 1718 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.320189 1718 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:49.320298 1709 ts_tablet_manager.cc:1428] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:49.320328 1716 ts_tablet_manager.cc:1428] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:49.320359 1726 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.320505 1726 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:49.320520 1717 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.320664 1717 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:49.321149 1712 ts_tablet_manager.cc:1428] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260807 00:01:49.388386 1718 raft_consensus.cc:482] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:49.388547 1718 raft_consensus.cc:504] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.389771 1718 leader_election.cc:288] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66:41645), 35b357b133444a8e9822da7932ff4c02 (127.30.204.67:40571)
I20260807 00:01:49.395182 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.390189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:33925 (local address 127.30.204.67:40571)
0807 00:01:49.390346 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.390351 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.390631 (+ 280us) server_negotiation.cc:407] Connection header received
0807 00:01:49.390816 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.390820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.390876 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.390965 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:49.391358 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.392192 (+ 834us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.393965 (+ 1773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.394591 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.394732 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.394842 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.394990 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.395066 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:01:49.395228 1778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.390085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40571 (local address 127.30.204.65:33925)
0807 00:01:49.390537 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.390548 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.390664 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.390983 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.390985 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.391011 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:49.391247 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.391255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.392306 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.392309 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.393841 (+ 1532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.393853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.394728 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.394733 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.394853 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.394880 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.394953 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.395035 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"mutex_wait_us":197,"spinlock_wait_cycles":14208,"thread_start_us":148,"threads_started":1}
I20260807 00:01:49.395398 1708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.390254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:43813 (local address 127.30.204.66:41645)
0807 00:01:49.390482 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.390486 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.390616 (+ 130us) server_negotiation.cc:407] Connection header received
0807 00:01:49.390816 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.390820 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.390876 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.390965 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:49.391364 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.392170 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.393965 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.394595 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.394819 (+ 224us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.394868 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.395184 (+ 316us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.395277 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141}
I20260807 00:01:49.395809 1777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.390081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:41645 (local address 127.30.204.65:43813)
0807 00:01:49.390492 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.390509 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.390671 (+ 162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.390982 (+ 311us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.390985 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.391011 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:49.391247 (+ 236us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.391255 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.392290 (+ 1035us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.392295 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.393841 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.393853 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.394938 (+ 1085us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.394942 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.395046 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.395069 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.395562 (+ 493us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.395655 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":330,"mutex_wait_us":135,"spinlock_wait_cycles":31360,"thread_start_us":154,"threads_started":1}
I20260807 00:01:49.395933 1609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f7bd27d80b45eab548ed29efd971e0" candidate_uuid: "513b5c5526b543dd9824b0f93a354001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b357b133444a8e9822da7932ff4c02" is_pre_election: true
I20260807 00:01:49.396101 1609 raft_consensus.cc:2454] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513b5c5526b543dd9824b0f93a354001 in term 0.
I20260807 00:01:49.396195 1379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f7bd27d80b45eab548ed29efd971e0" candidate_uuid: "513b5c5526b543dd9824b0f93a354001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907f199d94614b0e96a7435ec0dcf4ac" is_pre_election: true
I20260807 00:01:49.396343 1379 raft_consensus.cc:2454] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 513b5c5526b543dd9824b0f93a354001 in term 0.
I20260807 00:01:49.396490 1037 leader_election.cc:302] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 35b357b133444a8e9822da7932ff4c02, 513b5c5526b543dd9824b0f93a354001; no voters:
I20260807 00:01:49.396781 1718 raft_consensus.cc:2781] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:49.396839 1718 raft_consensus.cc:482] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:49.396878 1718 raft_consensus.cc:3037] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:49.397868 1718 raft_consensus.cc:504] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.398212 1718 leader_election.cc:288] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [CANDIDATE]: Term 1 election: Requested vote from peers 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66:41645), 35b357b133444a8e9822da7932ff4c02 (127.30.204.67:40571)
I20260807 00:01:49.398554 1379 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f7bd27d80b45eab548ed29efd971e0" candidate_uuid: "513b5c5526b543dd9824b0f93a354001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "907f199d94614b0e96a7435ec0dcf4ac"
I20260807 00:01:49.398690 1379 raft_consensus.cc:3037] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:49.398705 1609 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "41f7bd27d80b45eab548ed29efd971e0" candidate_uuid: "513b5c5526b543dd9824b0f93a354001" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "35b357b133444a8e9822da7932ff4c02"
I20260807 00:01:49.398838 1609 raft_consensus.cc:3037] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:49.399689 1379 raft_consensus.cc:2454] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513b5c5526b543dd9824b0f93a354001 in term 1.
I20260807 00:01:49.399837 1609 raft_consensus.cc:2454] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 513b5c5526b543dd9824b0f93a354001 in term 1.
I20260807 00:01:49.400440 1038 leader_election.cc:302] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 513b5c5526b543dd9824b0f93a354001, 907f199d94614b0e96a7435ec0dcf4ac; no voters:
I20260807 00:01:49.400691 1718 raft_consensus.cc:2781] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:49.400897 1718 raft_consensus.cc:686] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [term 1 LEADER]: Becoming Leader. State: Replica: 513b5c5526b543dd9824b0f93a354001, State: Running, Role: LEADER
I20260807 00:01:49.401142 1718 consensus_queue.cc:237] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } }
I20260807 00:01:49.402346 1779 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: TxnStatusTablet state changed. Reason: New leader 513b5c5526b543dd9824b0f93a354001. Latest consensus state: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.402478 1779 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:01:49.402773 1782 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 00:01:49.402972 1782 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 00:01:49.403736 890 catalog_manager.cc:5495] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 reported cstate change: term changed from 0 to 1, leader changed from <none> to 513b5c5526b543dd9824b0f93a354001 (127.30.204.65). New cstate: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } health_report { overall_health: UNKNOWN } } }
I20260807 00:01:49.424973 1770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.421084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44291 (local address 127.0.0.1:33152)
0807 00:01:49.421232 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.421249 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.421372 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.421731 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.421735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.421743 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:49.421855 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.421861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.422590 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.422594 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.423222 (+ 628us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.423231 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.423750 (+ 519us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.423754 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.423860 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.423911 (+ 51us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:49.424657 (+ 746us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:49.424677 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.424744 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.424846 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":29}
I20260807 00:01:49.424973 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.421341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33152 (local address 127.30.204.65:44291)
0807 00:01:49.421468 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.421472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.421487 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:01:49.421557 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.421559 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.421613 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.421701 (+ 88us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:49.421964 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.422481 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.423346 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.423638 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.423732 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.424065 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:49.424556 (+ 491us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:49.424665 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.424757 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.424823 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:01:49.428409 1609 raft_consensus.cc:1261] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Refusing update from remote peer 513b5c5526b543dd9824b0f93a354001: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:49.428478 1379 raft_consensus.cc:1261] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Refusing update from remote peer 513b5c5526b543dd9824b0f93a354001: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:49.428929 1779 consensus_queue.cc:1035] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:49.429140 1718 consensus_queue.cc:1035] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001 [LEADER]: Connected to new peer: Peer: permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:49.430812 1717 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: TxnStatusTablet state changed. Reason: New leader 513b5c5526b543dd9824b0f93a354001. Latest consensus state: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.430938 1717 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:49.431108 1726 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: TxnStatusTablet state changed. Reason: New leader 513b5c5526b543dd9824b0f93a354001. Latest consensus state: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.431219 1726 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:49.433141 1718 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.433269 1718 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:01:49.435189 1726 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.435315 1726 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 907f199d94614b0e96a7435ec0dcf4ac: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:49.439761 1718 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.439909 1718 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 513b5c5526b543dd9824b0f93a354001: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:01:49.440646 1717 tablet_replica.cc:439] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "513b5c5526b543dd9824b0f93a354001" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 } } peers { permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 } } peers { permanent_uuid: "35b357b133444a8e9822da7932ff4c02" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40571 } } }
I20260807 00:01:49.440783 1717 tablet_replica.cc:442] T 41f7bd27d80b45eab548ed29efd971e0 P 35b357b133444a8e9822da7932ff4c02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:49.444801 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.440273 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40702 (local address 127.30.204.67:40571)
0807 00:01:49.440730 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.440734 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.440748 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:01:49.440844 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.440847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.440917 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.441028 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:49.441354 (+ 326us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.442053 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.443138 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.443420 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.443550 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.443842 (+ 292us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:49.444384 (+ 542us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:49.444499 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.444575 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.444650 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":387}
I20260807 00:01:49.456496 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.452737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:59149 (local address 127.30.204.65:44291)
0807 00:01:49.452873 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.452878 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.452994 (+ 116us) server_negotiation.cc:407] Connection header received
0807 00:01:49.453163 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.453166 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.453216 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.453278 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:49.453877 (+ 599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.454402 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.455347 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.455634 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.455725 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.456029 (+ 304us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:49.456122 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:49.456184 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.456284 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.456340 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:01:49.456518 1797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.452573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44291 (local address 127.30.204.67:59149)
0807 00:01:49.452902 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.452918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.453053 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.453297 (+ 244us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.453299 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.453308 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:49.453443 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.453449 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.454510 (+ 1061us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.454514 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.455207 (+ 693us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.455219 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.455763 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.455768 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.455841 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.455894 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:49.456201 (+ 307us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:49.456208 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.456267 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.456355 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":118,"threads_started":1}
I20260807 00:01:49.464277 1379 raft_consensus.cc:1261] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Refusing update from remote peer 35b357b133444a8e9822da7932ff4c02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:49.464291 1153 raft_consensus.cc:1261] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Refusing update from remote peer 35b357b133444a8e9822da7932ff4c02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:49.464780 1760 consensus_queue.cc:1035] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "907f199d94614b0e96a7435ec0dcf4ac" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41645 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:49.465054 1717 consensus_queue.cc:1035] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "513b5c5526b543dd9824b0f93a354001" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44291 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:49.469753 1737 mvcc.cc:205] Tried to move back new op lower bound from 7315705485160566784 to 7315705484262363136. Current Snapshot: MvccSnapshot[applied={T|T < 7315705485160566784}]
I20260807 00:01:49.487798 1701 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.483633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:32779 (local address 127.30.204.67:40571)
0807 00:01:49.483763 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.483767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.484077 (+ 310us) server_negotiation.cc:407] Connection header received
0807 00:01:49.484202 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.484205 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.484253 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.484322 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:49.484694 (+ 372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.485328 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.486673 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.486933 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.487014 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.487343 (+ 329us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:49.487416 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:49.487503 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.487596 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.487677 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:01:49.487856 1806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.483577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40571 (local address 127.30.204.65:32779)
0807 00:01:49.483977 (+ 400us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:49.483993 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:49.484115 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:49.484343 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:49.484346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:49.484353 (+ 7us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:49.484495 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.484502 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.485438 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.485442 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.486557 (+ 1115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:49.486570 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.487043 (+ 473us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.487047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:49.487148 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.487216 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:49.487505 (+ 289us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:49.487511 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:01:49.487587 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 00:01:49.487676 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":131,"threads_started":1}
I20260807 00:01:49.513142 891 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:49.513341 891 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35344:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:49.515837 891 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 396de37ec63d477886fc2c07ffbf902e: Sending DeleteTablet for 3 replicas of tablet 621f70b1281c44dfa7bdb944a9758d87
I20260807 00:01:49.516489 1127 tablet_service.cc:1514] Processing DeleteTablet for tablet 621f70b1281c44dfa7bdb944a9758d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:49 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:33140
I20260807 00:01:49.516640 1353 tablet_service.cc:1514] Processing DeleteTablet for tablet 621f70b1281c44dfa7bdb944a9758d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:49 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:57620
I20260807 00:01:49.516964 1810 tablet_replica.cc:330] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: stopping tablet replica
I20260807 00:01:49.517155 1810 raft_consensus.cc:2229] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:49.517338 1810 raft_consensus.cc:2258] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:49.517355 1575 tablet_service.cc:1514] Processing DeleteTablet for tablet 621f70b1281c44dfa7bdb944a9758d87 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:49 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:40688
I20260807 00:01:49.517544 1750 tablet_replica.cc:330] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: stopping tablet replica
I20260807 00:01:49.517665 1750 raft_consensus.cc:2229] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:01:49.517959 1750 raft_consensus.cc:2258] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:49.518839 1750 ts_tablet_manager.cc:1905] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:49.519567 1811 tablet_replica.cc:330] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: stopping tablet replica
I20260807 00:01:49.519696 1811 raft_consensus.cc:2229] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:49.519817 1811 raft_consensus.cc:2258] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:49.520706 1811 ts_tablet_manager.cc:1905] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:49.522038 1750 ts_tablet_manager.cc:1918] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:49.522121 1750 log.cc:1196] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/621f70b1281c44dfa7bdb944a9758d87
I20260807 00:01:49.522372 1750 ts_tablet_manager.cc:1939] T 621f70b1281c44dfa7bdb944a9758d87 P 35b357b133444a8e9822da7932ff4c02: Deleting consensus metadata
I20260807 00:01:49.523166 873 catalog_manager.cc:4841] TS 35b357b133444a8e9822da7932ff4c02 (127.30.204.67:40571): tablet 621f70b1281c44dfa7bdb944a9758d87 (table test-table [id=8cc086ec045c4d4e9c7b7a2ed3fba96d]) successfully deleted
I20260807 00:01:49.524250 1810 ts_tablet_manager.cc:1905] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:49.524289 1811 ts_tablet_manager.cc:1918] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:49.524379 1811 log.cc:1196] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/621f70b1281c44dfa7bdb944a9758d87
I20260807 00:01:49.524641 1811 ts_tablet_manager.cc:1939] T 621f70b1281c44dfa7bdb944a9758d87 P 907f199d94614b0e96a7435ec0dcf4ac: Deleting consensus metadata
I20260807 00:01:49.525424 874 catalog_manager.cc:4841] TS 907f199d94614b0e96a7435ec0dcf4ac (127.30.204.66:41645): tablet 621f70b1281c44dfa7bdb944a9758d87 (table test-table [id=8cc086ec045c4d4e9c7b7a2ed3fba96d]) successfully deleted
I20260807 00:01:49.527294 1810 ts_tablet_manager.cc:1918] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:49.527379 1810 log.cc:1196] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/621f70b1281c44dfa7bdb944a9758d87
I20260807 00:01:49.527654 1810 ts_tablet_manager.cc:1939] T 621f70b1281c44dfa7bdb944a9758d87 P 513b5c5526b543dd9824b0f93a354001: Deleting consensus metadata
I20260807 00:01:49.528471 869 catalog_manager.cc:4841] TS 513b5c5526b543dd9824b0f93a354001 (127.30.204.65:44291): tablet 621f70b1281c44dfa7bdb944a9758d87 (table test-table [id=8cc086ec045c4d4e9c7b7a2ed3fba96d]) successfully deleted
Aug 07 00:01:49 dist-test-slave-pww0 krb5kdc[821](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060908, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:01:49.540481 1706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.518984 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33156 (local address 127.30.204.65:44291)
0807 00:01:49.523087 (+ 4103us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.523092 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.527871 (+ 4779us) server_negotiation.cc:407] Connection header received
0807 00:01:49.528078 (+ 207us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.528082 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.528137 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.528193 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:49.529270 (+ 1077us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.530048 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.531271 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.531635 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.531738 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.535690 (+ 3952us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:49.535700 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:49.535716 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:49.535750 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:49.537901 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:49.538516 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:49.538524 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:49.538526 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:49.538694 (+ 168us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:49.539018 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:49.539022 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:49.539024 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:49.539370 (+ 346us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:49.539660 (+ 290us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.540140 (+ 480us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.540280 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":4002}
W20260807 00:01:49.541731 1219 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:33156
I20260807 00:01:49.553527 1768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:49.543682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35354 (local address 127.30.204.126:37265)
0807 00:01:49.543814 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:49.543818 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:49.544313 (+ 495us) server_negotiation.cc:407] Connection header received
0807 00:01:49.544415 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:49.544419 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:49.544477 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:49.544622 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:49.545393 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.546229 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.547420 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:49.547846 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:49.547968 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:49.549603 (+ 1635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:49.549607 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:49.549610 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:49.549641 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:49.551592 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:49.552093 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:49.552097 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:49.552100 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:49.552207 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:49.552487 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:49.552490 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:49.552492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:49.552760 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:49.552901 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:49.553172 (+ 271us) server_negotiation.cc:299] Negotiation successful
0807 00:01:49.553329 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 00:01:49.554347 891 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35354
I20260807 00:01:49.556136 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1010
I20260807 00:01:49.569465 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1257
I20260807 00:01:49.597486 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1473
I20260807 00:01:49.612771 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 841
2026-08-07T00:01:49Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (3264 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:01:49 dist-test-slave-pww0 krb5kdc[1822](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:01:49 dist-test-slave-pww0 krb5kdc[1822](info): set up 2 sockets
Aug 07 00:01:49 dist-test-slave-pww0 krb5kdc[1822](info): commencing operation
krb5kdc: starting...
W20260807 00:01:50.790416 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.005s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:01:50 dist-test-slave-pww0 krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:01:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:01:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:01:50.938853 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:46713
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:46713
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:51.070415 1841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:51.070746 1841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:51.070807 1841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:51.075444 1841 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:01:51.075588 1841 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 00:01:51.075623 1841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:51.075649 1841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:51.075672 1841 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:01:51.075703 1841 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:01:51.081743 1841 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:46713
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:46713
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:51.083015 1841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:51.084210 1841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:51.093535 1848 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:51.093542 1852 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:51.093542 1849 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:51.094794 1841 server_base.cc:1034] running on GCE node
I20260807 00:01:51.095573 1841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:51.096935 1841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:51.098147 1841 hybrid_clock.cc:648] HybridClock initialized: now 1786060911098111 us; error 51 us; skew 500 ppm
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:51.101353 1841 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:01:51.102602 1841 webserver.cc:460] Webserver started at http://127.30.204.126:40705/ using document root <none> and password file <none>
I20260807 00:01:51.103229 1841 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:51.103307 1841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:51.103598 1841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:51.106009 1841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "4f9f34a7db374e3eb11059ecd683804f"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "3e825954cad761a1a6627a53f67f9b32"
server_key_iv: "3571d2e4c218a27f2abec4050fcf69e9"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.106943 1841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "4f9f34a7db374e3eb11059ecd683804f"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "3e825954cad761a1a6627a53f67f9b32"
server_key_iv: "3571d2e4c218a27f2abec4050fcf69e9"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.111583 1841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 00:01:51.114500 1867 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.116104 1841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:51.116250 1841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "4f9f34a7db374e3eb11059ecd683804f"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "3e825954cad761a1a6627a53f67f9b32"
server_key_iv: "3571d2e4c218a27f2abec4050fcf69e9"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.116393 1841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:51.130777 1841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:51.134241 1841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:51.134454 1841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:51.143220 1841 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:46713
I20260807 00:01:51.143270 1959 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:46713 every 8 connection(s)
I20260807 00:01:51.144312 1841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:01:51.147337 1960 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:51.147819 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1841
I20260807 00:01:51.147913 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:01:51.148175 31537 external_mini_cluster.cc:1428] Setting key 14a8737ee0fd4b8b8c485079dc55b118
I20260807 00:01:51.154116 1960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Bootstrap starting.
I20260807 00:01:51.157759 1960 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Neither blocks nor log segments found. Creating new log.
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060910, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:51.158658 1960 log.cc:824] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:51.161477 1960 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: No bootstrap required, opened a new log
I20260807 00:01:51.165181 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.149701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53750 (local address 127.30.204.126:46713)
0807 00:01:51.150146 (+ 445us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:51.150164 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:51.150202 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:01:51.150896 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:51.150915 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:51.151280 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:51.151631 (+ 351us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:51.152557 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.153619 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.154842 (+ 1223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.155390 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.155601 (+ 211us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.158732 (+ 3131us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:51.158739 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:51.158756 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:51.158796 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:51.161585 (+ 2789us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.162110 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.162117 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.162124 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.162277 (+ 153us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.162552 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.162555 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.162557 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.163131 (+ 574us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:51.163325 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:51.163699 (+ 374us) server_negotiation.cc:299] Negotiation successful
0807 00:01:51.164058 (+ 359us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":292,"thread_start_us":124,"threads_started":1}
I20260807 00:01:51.165745 1960 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9f34a7db374e3eb11059ecd683804f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46713 } }
I20260807 00:01:51.166067 1960 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:51.166139 1960 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4f9f34a7db374e3eb11059ecd683804f, State: Initialized, Role: FOLLOWER
I20260807 00:01:51.166632 1960 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9f34a7db374e3eb11059ecd683804f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46713 } }
I20260807 00:01:51.166785 1960 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:01:51.166869 1960 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:01:51.166994 1960 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:51.168282 1960 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9f34a7db374e3eb11059ecd683804f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46713 } }
I20260807 00:01:51.168653 1960 leader_election.cc:302] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4f9f34a7db374e3eb11059ecd683804f; no voters:
I20260807 00:01:51.169009 1960 leader_election.cc:288] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:01:51.169181 1968 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:51.169423 1968 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [term 1 LEADER]: Becoming Leader. State: Replica: 4f9f34a7db374e3eb11059ecd683804f, State: Running, Role: LEADER
I20260807 00:01:51.169749 1968 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9f34a7db374e3eb11059ecd683804f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46713 } }
I20260807 00:01:51.170426 1960 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:01:51.171993 1969 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4f9f34a7db374e3eb11059ecd683804f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9f34a7db374e3eb11059ecd683804f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46713 } } }
I20260807 00:01:51.172142 1969 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:51.172439 1970 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4f9f34a7db374e3eb11059ecd683804f. Latest consensus state: current_term: 1 leader_uuid: "4f9f34a7db374e3eb11059ecd683804f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4f9f34a7db374e3eb11059ecd683804f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46713 } } }
I20260807 00:01:51.172518 1970 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f [sys.catalog]: This master's current role is: LEADER
W20260807 00:01:51.175519 1993 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:01:51.175623 1993 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:01:51.175683 1985 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:01:51.176410 1985 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:01:51.183068 1985 catalog_manager.cc:1347] Generated new cluster ID: d9de544be8924c96b395302da9950022
I20260807 00:01:51.183190 1985 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:01:51.212841 1985 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:01:51.214303 1985 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:01:51.227233 1985 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Generated new TSK 0
I20260807 00:01:51.228287 1985 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:01:51.263592 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:46713
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:51.393275 2002 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:51.393589 2002 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:51.393639 2002 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:51.397912 2002 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:51.397996 2002 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:51.398037 2002 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 00:01:51.398051 2002 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:01:51.398087 2002 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:01:51.403761 2002 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2002
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:51.404979 2002 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:51.406100 2002 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:51.416541 2013 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:51.416531 2015 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:51.416528 2012 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:51.417116 2002 server_base.cc:1034] running on GCE node
I20260807 00:01:51.417606 2002 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:51.418350 2002 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:51.419531 2002 hybrid_clock.cc:648] HybridClock initialized: now 1786060911419503 us; error 43 us; skew 500 ppm
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:51.422624 2002 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:01:51.423722 2002 webserver.cc:460] Webserver started at http://127.30.204.65:33499/ using document root <none> and password file <none>
I20260807 00:01:51.424336 2002 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:51.424412 2002 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:51.424618 2002 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:51.426783 2002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "073070cedeef47d3a6e3db81e9c69c02"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "ac54fdf454323d5b737444886422c4ef"
server_key_iv: "4ff0f52ddefbe47f8eb77dd0c771ca9d"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.427381 2002 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "073070cedeef47d3a6e3db81e9c69c02"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "ac54fdf454323d5b737444886422c4ef"
server_key_iv: "4ff0f52ddefbe47f8eb77dd0c771ca9d"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.431491 2002 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:01:51.434311 2026 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.435851 2002 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:51.435979 2002 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "073070cedeef47d3a6e3db81e9c69c02"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "ac54fdf454323d5b737444886422c4ef"
server_key_iv: "4ff0f52ddefbe47f8eb77dd0c771ca9d"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.436116 2002 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:51.465437 2002 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:51.468921 2002 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:51.469120 2002 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:51.470857 2002 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:51.470919 2002 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.470978 2002 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:51.471012 2002 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:51.482295 2002 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:33965
I20260807 00:01:51.482939 2213 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:33965 every 8 connection(s)
I20260807 00:01:51.483507 2002 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:01:51.486442 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.471804 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:46423 (local address 127.30.204.126:46713)
0807 00:01:51.471968 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:51.471974 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:51.473104 (+ 1130us) server_negotiation.cc:407] Connection header received
0807 00:01:51.474523 (+ 1419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:51.474527 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:51.474594 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:51.474695 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:51.476143 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.476697 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.478584 (+ 1887us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.478991 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.479103 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.482433 (+ 3330us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:51.482438 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:51.482441 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:51.482477 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:51.484104 (+ 1627us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.484697 (+ 593us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.484702 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.484703 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.484798 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.485230 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.485234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.485239 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.485463 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:51.485554 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:51.486159 (+ 605us) server_negotiation.cc:299] Negotiation successful
0807 00:01:51.486309 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:01:51.487386 2038 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.472334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46713 (local address 127.30.204.65:46423)
0807 00:01:51.472896 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:51.472963 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:51.474134 (+ 1171us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:51.474894 (+ 760us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:51.474902 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:51.475295 (+ 393us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:51.475973 (+ 678us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.475992 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.476834 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.476838 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.478391 (+ 1553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.478410 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.479172 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.479176 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.479391 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.480200 (+ 809us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:51.480226 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:51.482183 (+ 1957us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:51.484266 (+ 2083us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.484272 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.484289 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.484583 (+ 294us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.484923 (+ 340us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.484926 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.484928 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.485096 (+ 168us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.485577 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:51.485584 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:51.485895 (+ 311us) client_negotiation.cc:769] Sending connection context
0807 00:01:51.486084 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 00:01:51.486393 (+ 309us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":310,"thread_start_us":143,"threads_started":1}
I20260807 00:01:51.489215 2214 heartbeater.cc:347] Connected to a master server at 127.30.204.126:46713
I20260807 00:01:51.489447 2214 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:51.490101 2214 heartbeater.cc:511] Master 127.30.204.126:46713 requested a full tablet report, sending...
I20260807 00:01:51.491986 1892 ts_manager.cc:194] Registered new tserver with Master: 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965)
I20260807 00:01:51.493163 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2002
I20260807 00:01:51.493269 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:01:51.493259 1892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:46423
I20260807 00:01:51.493556 31537 external_mini_cluster.cc:1428] Setting key 867ed7de7e181771595e6ea24e08eec5
I20260807 00:01:51.501797 2229 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.491663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46713 (local address 127.30.204.65:39225)
0807 00:01:51.494071 (+ 2408us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:51.494086 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:51.494214 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:51.494489 (+ 275us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:51.494492 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:51.494724 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:51.494834 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.494841 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.495788 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.495792 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.496444 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.496455 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.496895 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.496899 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.496991 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.497549 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:51.497562 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:51.497931 (+ 369us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:51.499990 (+ 2059us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.499994 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.499995 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.500256 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.500578 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.500581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.500582 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.500683 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.501171 (+ 488us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:51.501174 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:51.501365 (+ 191us) client_negotiation.cc:769] Sending connection context
0807 00:01:51.501465 (+ 100us) client_negotiation.cc:240] Negotiation successful
0807 00:01:51.501624 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2302,"thread_start_us":102,"threads_started":1}
I20260807 00:01:51.501937 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.491752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:39225 (local address 127.30.204.126:46713)
0807 00:01:51.491892 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:51.491897 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:51.494184 (+ 2287us) server_negotiation.cc:407] Connection header received
0807 00:01:51.494339 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:51.494342 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:51.494386 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:51.494452 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:51.494954 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.495624 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.496555 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.496786 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.496876 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.498076 (+ 1200us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:51.498078 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:51.498080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:51.498105 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:51.499855 (+ 1750us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.500373 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.500377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.500379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.500480 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.500777 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.500780 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.500781 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.501023 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:51.501123 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:51.501592 (+ 469us) server_negotiation.cc:299] Negotiation successful
0807 00:01:51.501774 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:01:51.524928 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:46713
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:51.644162 2238 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:51.644485 2238 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:51.644558 2238 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:51.648917 2238 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:51.649000 2238 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:51.649060 2238 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 00:01:51.649085 2238 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:01:51.649144 2238 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:01:51.654915 2238 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2238
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:51.656279 2238 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:51.657454 2238 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:51.668113 2251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:51.668113 2244 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:51.668143 2246 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:51.668398 2238 server_base.cc:1034] running on GCE node
I20260807 00:01:51.668891 2238 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:51.669656 2238 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:51.670847 2238 hybrid_clock.cc:648] HybridClock initialized: now 1786060911670829 us; error 42 us; skew 500 ppm
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:51.674095 2238 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:01:51.675123 2238 webserver.cc:460] Webserver started at http://127.30.204.66:41097/ using document root <none> and password file <none>
I20260807 00:01:51.675769 2238 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:51.675841 2238 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:51.676038 2238 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:51.678510 2238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "e914d5969c77458685dfa4e7f09513f1"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "b1bc1c9b077d79c9eb2c4372735cd158"
server_key_iv: "63c0601c4972fda76f5ec1b7cf7a70ee"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.679124 2238 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "e914d5969c77458685dfa4e7f09513f1"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "b1bc1c9b077d79c9eb2c4372735cd158"
server_key_iv: "63c0601c4972fda76f5ec1b7cf7a70ee"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.683517 2238 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:01:51.686522 2261 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.688120 2238 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:51.688256 2238 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "e914d5969c77458685dfa4e7f09513f1"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "b1bc1c9b077d79c9eb2c4372735cd158"
server_key_iv: "63c0601c4972fda76f5ec1b7cf7a70ee"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.688378 2238 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:51.702394 2238 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:51.706038 2238 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:51.706262 2238 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:51.707916 2238 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:51.707975 2238 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.708033 2238 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:51.708070 2238 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.718134 2238 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:35419
I20260807 00:01:51.718246 2457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:35419 every 8 connection(s)
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:51.719172 2238 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:01:51.723089 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.708788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:59427 (local address 127.30.204.126:46713)
0807 00:01:51.708945 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:51.708950 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:51.709937 (+ 987us) server_negotiation.cc:407] Connection header received
0807 00:01:51.711190 (+ 1253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:51.711195 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:51.711267 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:51.711376 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:51.713256 (+ 1880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.714096 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.715748 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.716103 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.716242 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.719249 (+ 3007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:51.719254 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:51.719257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:51.719292 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:51.720973 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.721540 (+ 567us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.721543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.721545 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.721669 (+ 124us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.722029 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.722032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.722034 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.722256 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:51.722348 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:51.722839 (+ 491us) server_negotiation.cc:299] Negotiation successful
0807 00:01:51.722959 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:01:51.723583 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2238
I20260807 00:01:51.723686 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:01:51.723958 31537 external_mini_cluster.cc:1428] Setting key 9b9636b12d5753e3c10669585976fb72
I20260807 00:01:51.724156 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.709224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46713 (local address 127.30.204.66:59427)
0807 00:01:51.709755 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:51.709811 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:51.710862 (+ 1051us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:51.711754 (+ 892us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:51.711762 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:51.712277 (+ 515us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:51.713034 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.713059 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.714275 (+ 1216us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.714280 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.715604 (+ 1324us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.715615 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.716296 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.716301 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.716539 (+ 238us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.717312 (+ 773us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:51.717331 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:51.719022 (+ 1691us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:51.721100 (+ 2078us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.721106 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.721119 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.721421 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.721773 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.721775 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.721776 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.721915 (+ 139us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.722356 (+ 441us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:51.722361 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:51.722623 (+ 262us) client_negotiation.cc:769] Sending connection context
0807 00:01:51.722804 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 00:01:51.723101 (+ 297us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":118,"threads_started":1}
I20260807 00:01:51.725550 2458 heartbeater.cc:347] Connected to a master server at 127.30.204.126:46713
I20260807 00:01:51.725735 2458 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:51.726383 2458 heartbeater.cc:511] Master 127.30.204.126:46713 requested a full tablet report, sending...
I20260807 00:01:51.727664 1892 ts_manager.cc:194] Registered new tserver with Master: e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419)
I20260807 00:01:51.728454 1892 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:59427
I20260807 00:01:51.735522 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.727976 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:36015 (local address 127.30.204.126:46713)
0807 00:01:51.728094 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:51.728098 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:51.728362 (+ 264us) server_negotiation.cc:407] Connection header received
0807 00:01:51.728465 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:51.728468 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:51.728521 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:51.728596 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:51.729307 (+ 711us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.729832 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.730695 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.730918 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.731008 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.732301 (+ 1293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:51.732305 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:51.732307 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:51.732334 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:51.733789 (+ 1455us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.734221 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.734225 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.734227 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.734315 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.734577 (+ 262us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.734581 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.734582 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.734833 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:51.734927 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:51.735206 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 00:01:51.735327 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:01:51.735536 2469 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.727901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46713 (local address 127.30.204.66:36015)
0807 00:01:51.728262 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:51.728276 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:51.728383 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:51.728728 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:51.728731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:51.728968 (+ 237us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:51.729101 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.729109 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.729966 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.729969 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.730574 (+ 605us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.730583 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.731024 (+ 441us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.731028 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.731125 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.731725 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:51.731738 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:51.732136 (+ 398us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:51.733908 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.733911 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.733913 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.734124 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.734400 (+ 276us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.734404 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.734406 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.734493 (+ 87us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.734932 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:51.734935 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:51.735039 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 00:01:51.735130 (+ 91us) client_negotiation.cc:240] Negotiation successful
0807 00:01:51.735304 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":272,"mutex_wait_us":1,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:01:51.755136 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:46713
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44859
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:51.876101 2472 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:51.876410 2472 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:51.876463 2472 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:51.880668 2472 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:51.880748 2472 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:51.880795 2472 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260807 00:01:51.880810 2472 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:01:51.880863 2472 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:01:51.886610 2472 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44859
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:46713
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2472
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:51.887880 2472 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:51.889063 2472 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:51.899816 2481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:51.899940 2472 server_base.cc:1034] running on GCE node
W20260807 00:01:51.899894 2488 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:51.899816 2484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:51.900594 2472 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:51.901298 2472 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:51.902479 2472 hybrid_clock.cc:648] HybridClock initialized: now 1786060911902457 us; error 36 us; skew 500 ppm
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:51.905467 2472 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:01:51.906463 2472 webserver.cc:460] Webserver started at http://127.30.204.67:34241/ using document root <none> and password file <none>
I20260807 00:01:51.907047 2472 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:51.907120 2472 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:51.907446 2472 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:51.909780 2472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "3ec76df22e94404bbe72f3e130170b15"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "2809f5756f554c82a9ae4d4ca977a3bd"
server_key_iv: "de1035189b0c45081877485211308948"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.910383 2472 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "3ec76df22e94404bbe72f3e130170b15"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "2809f5756f554c82a9ae4d4ca977a3bd"
server_key_iv: "de1035189b0c45081877485211308948"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.914601 2472 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 00:01:51.917405 2498 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.918866 2472 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:51.919008 2472 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "3ec76df22e94404bbe72f3e130170b15"
format_stamp: "Formatted at 2026-08-07 00:01:51 on dist-test-slave-pww0"
server_key: "2809f5756f554c82a9ae4d4ca977a3bd"
server_key_iv: "de1035189b0c45081877485211308948"
server_key_version: "encryptionkey@0"
I20260807 00:01:51.919137 2472 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:51.963984 2472 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:51.967314 2472 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:51.967549 2472 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:51.969182 2472 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:51.969238 2472 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:51.969297 2472 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:51.969326 2472 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:01:51 dist-test-slave-pww0 krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:51.980126 2472 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:33945
I20260807 00:01:51.980194 2677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:33945 every 8 connection(s)
I20260807 00:01:51.981313 2472 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:01:51.983722 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2472
I20260807 00:01:51.983820 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:01:51.984117 31537 external_mini_cluster.cc:1428] Setting key 0223df5f457f66a883846766835d8997
I20260807 00:01:51.984431 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.970190 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:38153 (local address 127.30.204.126:46713)
0807 00:01:51.970406 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:51.970411 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:51.971633 (+ 1222us) server_negotiation.cc:407] Connection header received
0807 00:01:51.972879 (+ 1246us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:51.972883 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:51.972951 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:51.973041 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:51.974674 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.975549 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.976877 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.977227 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.977354 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.980470 (+ 3116us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:51.980475 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:51.980477 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:51.980512 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:51.982273 (+ 1761us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.982787 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.982790 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.982793 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.982877 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.983200 (+ 323us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.983204 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.983205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.983436 (+ 231us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:51.983555 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:51.984116 (+ 561us) server_negotiation.cc:299] Negotiation successful
0807 00:01:51.984263 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 00:01:51.985458 2511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.970813 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46713 (local address 127.30.204.67:38153)
0807 00:01:51.971448 (+ 635us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:51.971528 (+ 80us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:51.972593 (+ 1065us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:51.973288 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:51.973298 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:51.973750 (+ 452us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:51.974495 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.974520 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.975696 (+ 1176us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.975699 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.976738 (+ 1039us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.976751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.977426 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.977429 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.977606 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.978304 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:51.978329 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:51.980244 (+ 1915us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:51.982398 (+ 2154us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.982405 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.982420 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.982677 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.982963 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.982967 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.982969 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.983112 (+ 143us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.983554 (+ 442us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:51.983561 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:51.983849 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 00:01:51.984055 (+ 206us) client_negotiation.cc:240] Negotiation successful
0807 00:01:51.984409 (+ 354us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":343,"thread_start_us":177,"threads_started":1}
I20260807 00:01:51.987044 2678 heartbeater.cc:347] Connected to a master server at 127.30.204.126:46713
I20260807 00:01:51.987207 2678 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:51.987775 2678 heartbeater.cc:511] Master 127.30.204.126:46713 requested a full tablet report, sending...
I20260807 00:01:51.988948 1891 ts_manager.cc:194] Registered new tserver with Master: 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:51.989656 1891 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:38153
I20260807 00:01:51.996661 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.989947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:42143 (local address 127.30.204.126:46713)
0807 00:01:51.990070 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:51.990074 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:51.990102 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:01:51.990140 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:51.990143 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:51.990186 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:51.990284 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:51.990743 (+ 459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.991266 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.992151 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.992403 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.992489 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.993690 (+ 1201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:51.993694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:51.993697 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:51.993730 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:51.995053 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.995487 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.995491 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.995492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.995579 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:51.995838 (+ 259us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:51.995842 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:51.995843 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:51.996029 (+ 186us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:51.996117 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:51.996376 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 00:01:51.996489 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:01:51.996675 2684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:51.989391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46713 (local address 127.30.204.67:42143)
0807 00:01:51.989698 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:51.989711 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:51.989806 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:51.990307 (+ 501us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:51.990310 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:51.990523 (+ 213us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:51.990629 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.990635 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.991384 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.991388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.992015 (+ 627us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:51.992025 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:51.992508 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:51.992512 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:51.992603 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:51.993123 (+ 520us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:51.993138 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:51.993552 (+ 414us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:51.995160 (+ 1608us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.995163 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.995166 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.995372 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.995665 (+ 293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:51.995668 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:51.995670 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:51.995756 (+ 86us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:51.996119 (+ 363us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:51.996122 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:51.996223 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 00:01:51.996310 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:01:51.996469 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":86,"threads_started":1}
I20260807 00:01:51.998342 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:01:52 dist-test-slave-pww0 krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:01:52 dist-test-slave-pww0 krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060912, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:52.027634 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.018168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38650 (local address 127.30.204.126:46713)
0807 00:01:52.018315 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.018319 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.018331 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:01:52.018374 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.018377 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.018418 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.018478 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:52.019257 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.019909 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.020947 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.021193 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.021294 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.024135 (+ 2841us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:52.024138 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:52.024141 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:52.024178 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:52.025711 (+ 1533us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:52.026247 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:52.026251 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:52.026253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:52.026367 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:52.026727 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:52.026730 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:52.026731 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:52.026961 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:52.027069 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.027337 (+ 268us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.027448 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:01:52.031080 1891 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:01:52.034545 1891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:52.050663 2702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.044266 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:33965 (local address 127.0.0.1:57346)
0807 00:01:52.044878 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.044892 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.045037 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.046043 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.046046 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.046063 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.046151 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.046156 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.047717 (+ 1561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.047721 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.049053 (+ 1332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.049062 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.050036 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.050040 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.050156 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.050209 (+ 53us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.050344 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.050487 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":544,"mutex_wait_us":32,"thread_start_us":123,"threads_started":1}
I20260807 00:01:52.050690 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.045311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57346 (local address 127.30.204.65:33965)
0807 00:01:52.045606 (+ 295us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.045612 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.045635 (+ 23us) server_negotiation.cc:407] Connection header received
0807 00:01:52.045709 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.045713 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.045900 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.046314 (+ 414us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.046451 (+ 137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.047569 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.049218 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.049909 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.050009 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.050179 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.050408 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.050527 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":183,"thread_start_us":76,"threads_started":1}
I20260807 00:01:52.051080 2707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.044841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:33945 (local address 127.0.0.1:46122)
0807 00:01:52.045989 (+ 1148us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.046003 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.046171 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.046722 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.046724 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.046741 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.046830 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.046834 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.047952 (+ 1118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.047955 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.049455 (+ 1500us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.049465 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.050673 (+ 1208us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.050677 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.050773 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.050798 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.050867 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.050944 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1087,"mutex_wait_us":311,"thread_start_us":109,"threads_started":1}
I20260807 00:01:52.051244 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.044197 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37326 (local address 127.30.204.66:35419)
0807 00:01:52.044734 (+ 537us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.044746 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.044970 (+ 224us) server_negotiation.cc:407] Connection header received
0807 00:01:52.045263 (+ 293us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.045270 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.045474 (+ 204us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.045615 (+ 141us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.046186 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.047501 (+ 1315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.049037 (+ 1536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.049932 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.050038 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.050210 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.050622 (+ 412us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.051096 (+ 474us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":417,"thread_start_us":127,"threads_started":1}
I20260807 00:01:52.051558 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.045036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46122 (local address 127.30.204.67:33945)
0807 00:01:52.045845 (+ 809us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.045859 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.046086 (+ 227us) server_negotiation.cc:407] Connection header received
0807 00:01:52.046319 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.046324 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.046486 (+ 162us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.046581 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.046942 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.047833 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.049571 (+ 1738us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.050351 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.050453 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.050607 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.051270 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.051375 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":700,"thread_start_us":131,"threads_started":1}
I20260807 00:01:52.051693 2705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.044041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35419 (local address 127.0.0.1:37326)
0807 00:01:52.044808 (+ 767us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.044870 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.045055 (+ 185us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.045853 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.045859 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.045884 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.046057 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.046064 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.047628 (+ 1564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.047632 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.048919 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.048928 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.050362 (+ 1434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.050365 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.050463 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.050485 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.051436 (+ 951us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.051552 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"mutex_wait_us":70,"thread_start_us":81,"threads_started":1}
I20260807 00:01:52.054266 2102 tablet_service.cc:1467] Processing CreateTablet for tablet 4ecfe42b92bb4514829d22e8c0a125c5 (DEFAULT_TABLE table=test-table [id=6873dbc077f949708aadba003c521b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:52.054278 2566 tablet_service.cc:1467] Processing CreateTablet for tablet 4ecfe42b92bb4514829d22e8c0a125c5 (DEFAULT_TABLE table=test-table [id=6873dbc077f949708aadba003c521b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:52.055598 2102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ecfe42b92bb4514829d22e8c0a125c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.055598 2566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ecfe42b92bb4514829d22e8c0a125c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.058475 2348 tablet_service.cc:1467] Processing CreateTablet for tablet 4ecfe42b92bb4514829d22e8c0a125c5 (DEFAULT_TABLE table=test-table [id=6873dbc077f949708aadba003c521b08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:52.059578 2348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4ecfe42b92bb4514829d22e8c0a125c5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.063660 2714 tablet_bootstrap.cc:492] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Bootstrap starting.
I20260807 00:01:52.064503 2715 tablet_bootstrap.cc:492] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Bootstrap starting.
I20260807 00:01:52.066390 2715 tablet_bootstrap.cc:654] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.066479 2714 tablet_bootstrap.cc:654] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.067251 2715 log.cc:824] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:52.067345 2714 log.cc:824] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:52.067323 2717 tablet_bootstrap.cc:492] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Bootstrap starting.
I20260807 00:01:52.069576 2715 tablet_bootstrap.cc:492] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: No bootstrap required, opened a new log
I20260807 00:01:52.069737 2714 tablet_bootstrap.cc:492] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: No bootstrap required, opened a new log
I20260807 00:01:52.069799 2715 ts_tablet_manager.cc:1397] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Time spent bootstrapping tablet: real 0.005s user 0.003s sys 0.000s
I20260807 00:01:52.069901 2717 tablet_bootstrap.cc:654] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.069940 2714 ts_tablet_manager.cc:1397] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Time spent bootstrapping tablet: real 0.007s user 0.002s sys 0.002s
I20260807 00:01:52.070789 2717 log.cc:824] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Log is configured to *not* fsync() on all Append() calls
I20260807 00:01:52.073218 2717 tablet_bootstrap.cc:492] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: No bootstrap required, opened a new log
I20260807 00:01:52.073441 2717 ts_tablet_manager.cc:1397] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:01:52.073789 2714 raft_consensus.cc:348] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.073802 2715 raft_consensus.cc:348] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.074038 2714 raft_consensus.cc:374] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.074071 2715 raft_consensus.cc:374] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.074102 2714 raft_consensus.cc:729] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073070cedeef47d3a6e3db81e9c69c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.074120 2715 raft_consensus.cc:729] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec76df22e94404bbe72f3e130170b15, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.074544 2714 consensus_queue.cc:260] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.074613 2715 consensus_queue.cc:260] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.075242 2214 heartbeater.cc:503] Master 127.30.204.126:46713 was elected leader, sending a full tablet report...
I20260807 00:01:52.075657 2714 ts_tablet_manager.cc:1428] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Time spent starting tablet: real 0.006s user 0.005s sys 0.001s
I20260807 00:01:52.075659 2715 ts_tablet_manager.cc:1428] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:01:52.076009 2678 heartbeater.cc:503] Master 127.30.204.126:46713 was elected leader, sending a full tablet report...
I20260807 00:01:52.077600 2717 raft_consensus.cc:348] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.077910 2717 raft_consensus.cc:374] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.077965 2717 raft_consensus.cc:729] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.078477 2717 consensus_queue.cc:260] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.079679 2717 ts_tablet_manager.cc:1428] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:01:52.079778 2458 heartbeater.cc:503] Master 127.30.204.126:46713 was elected leader, sending a full tablet report...
I20260807 00:01:52.185206 2722 raft_consensus.cc:482] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:52.185462 2722 raft_consensus.cc:504] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.186848 2722 leader_election.cc:288] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:52.188941 2727 raft_consensus.cc:482] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:52.189152 2727 raft_consensus.cc:504] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.190467 2727 leader_election.cc:288] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:52.191699 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.187310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:37325 (local address 127.30.204.66:35419)
0807 00:01:52.187475 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.187480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.187847 (+ 367us) server_negotiation.cc:407] Connection header received
0807 00:01:52.188039 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.188041 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.188089 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.188162 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.188455 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.189116 (+ 661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.190368 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.190921 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.191048 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.191111 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.191447 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.191559 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 00:01:52.192193 2734 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.187188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35419 (local address 127.30.204.65:37325)
0807 00:01:52.187755 (+ 567us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.187771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.187897 (+ 126us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.188181 (+ 284us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.188184 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.188201 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.188333 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.188341 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.189218 (+ 877us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.189221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.190220 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.190231 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.191174 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.191179 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.191287 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.191313 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.191914 (+ 601us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.192000 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":439,"mutex_wait_us":77,"thread_start_us":127,"threads_started":1}
I20260807 00:01:52.192641 2708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.187310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:43651 (local address 127.30.204.67:33945)
0807 00:01:52.187475 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.187480 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.187773 (+ 293us) server_negotiation.cc:407] Connection header received
0807 00:01:52.187969 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.187972 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.188018 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.188089 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.188359 (+ 270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.189017 (+ 658us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.191327 (+ 2310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.191880 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.191990 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.192028 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.192427 (+ 399us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.192507 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:01:52.193076 2378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecfe42b92bb4514829d22e8c0a125c5" candidate_uuid: "073070cedeef47d3a6e3db81e9c69c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e914d5969c77458685dfa4e7f09513f1" is_pre_election: true
I20260807 00:01:52.193265 2731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.187188 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:33945 (local address 127.30.204.65:43651)
0807 00:01:52.187669 (+ 481us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.187685 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.187814 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.188106 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.188108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.188124 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.188253 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.188258 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.189164 (+ 906us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.189167 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.191181 (+ 2014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.191192 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.192140 (+ 948us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.192144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.192278 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.192305 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.193009 (+ 704us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.193095 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":400,"mutex_wait_us":908,"thread_start_us":139,"threads_started":1}
I20260807 00:01:52.193354 2378 raft_consensus.cc:2454] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073070cedeef47d3a6e3db81e9c69c02 in term 0.
I20260807 00:01:52.193888 2030 leader_election.cc:302] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073070cedeef47d3a6e3db81e9c69c02, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:52.193922 2598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecfe42b92bb4514829d22e8c0a125c5" candidate_uuid: "073070cedeef47d3a6e3db81e9c69c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15" is_pre_election: true
I20260807 00:01:52.194229 2722 raft_consensus.cc:2781] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:52.194258 2598 raft_consensus.cc:2454] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 073070cedeef47d3a6e3db81e9c69c02 in term 0.
I20260807 00:01:52.194345 2722 raft_consensus.cc:482] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:52.194401 2722 raft_consensus.cc:3037] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.195629 2274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.190653 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:33965 (local address 127.30.204.66:45135)
0807 00:01:52.190792 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.190809 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.190901 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.191891 (+ 990us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.191895 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.191918 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.192034 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.192042 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.193005 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.193008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.194184 (+ 1176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.194198 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.195046 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.195050 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.195231 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.195264 (+ 33us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.195354 (+ 90us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.195434 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":57}
I20260807 00:01:52.195650 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.191129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:45135 (local address 127.30.204.65:33965)
0807 00:01:52.191252 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.191258 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.191271 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:01:52.191311 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.191314 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.191361 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.191433 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.192160 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.192878 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.194335 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.194931 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.195035 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.195071 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.195399 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.195519 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":75,"server-negotiator.queue_time_us":53}
I20260807 00:01:52.196038 2722 raft_consensus.cc:504] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.196277 2108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecfe42b92bb4514829d22e8c0a125c5" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02" is_pre_election: true
I20260807 00:01:52.196467 2722 leader_election.cc:288] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [CANDIDATE]: Term 1 election: Requested vote from peers e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:52.196518 2108 raft_consensus.cc:2379] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e914d5969c77458685dfa4e7f09513f1 in current term 1: Already voted for candidate 073070cedeef47d3a6e3db81e9c69c02 in this term.
I20260807 00:01:52.196946 2378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecfe42b92bb4514829d22e8c0a125c5" candidate_uuid: "073070cedeef47d3a6e3db81e9c69c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e914d5969c77458685dfa4e7f09513f1"
I20260807 00:01:52.197079 2378 raft_consensus.cc:3037] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.197095 2598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecfe42b92bb4514829d22e8c0a125c5" candidate_uuid: "073070cedeef47d3a6e3db81e9c69c02" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15"
I20260807 00:01:52.197239 2598 raft_consensus.cc:3037] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.198186 2735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.190888 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:33945 (local address 127.30.204.66:51429)
0807 00:01:52.191361 (+ 473us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.191374 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.191490 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.192731 (+ 1241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.192735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.192756 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.192880 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.192886 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.195390 (+ 2504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.195394 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.196549 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.196561 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.197442 (+ 881us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.197446 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.197548 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.197588 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.197672 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.197754 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"thread_start_us":113,"threads_started":1}
I20260807 00:01:52.198200 2738 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.191635 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:51429 (local address 127.30.204.67:33945)
0807 00:01:52.192381 (+ 746us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.192392 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.192408 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:01:52.192457 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.192460 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.192512 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.192612 (+ 100us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.193002 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.195263 (+ 2261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.196692 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.197283 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.197414 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.197468 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.197702 (+ 234us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.197782 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":663,"thread_start_us":123,"threads_started":1}
I20260807 00:01:52.198518 2378 raft_consensus.cc:2454] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073070cedeef47d3a6e3db81e9c69c02 in term 1.
I20260807 00:01:52.198632 2598 raft_consensus.cc:2454] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 073070cedeef47d3a6e3db81e9c69c02 in term 1.
I20260807 00:01:52.198643 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4ecfe42b92bb4514829d22e8c0a125c5" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15" is_pre_election: true
I20260807 00:01:52.198828 2594 raft_consensus.cc:2379] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate e914d5969c77458685dfa4e7f09513f1 in current term 1: Already voted for candidate 073070cedeef47d3a6e3db81e9c69c02 in this term.
I20260807 00:01:52.198913 2030 leader_election.cc:302] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073070cedeef47d3a6e3db81e9c69c02, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:52.199142 2722 raft_consensus.cc:2781] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:52.199409 2722 raft_consensus.cc:686] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 1 LEADER]: Becoming Leader. State: Replica: 073070cedeef47d3a6e3db81e9c69c02, State: Running, Role: LEADER
I20260807 00:01:52.199381 2262 leader_election.cc:302] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: e914d5969c77458685dfa4e7f09513f1; no voters: 073070cedeef47d3a6e3db81e9c69c02, 3ec76df22e94404bbe72f3e130170b15
I20260807 00:01:52.199725 2727 raft_consensus.cc:2726] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260807 00:01:52.199837 2722 consensus_queue.cc:237] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.203356 1891 catalog_manager.cc:5495] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 reported cstate change: term changed from 0 to 1, leader changed from <none> to 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65). New cstate: current_term: 1 leader_uuid: "073070cedeef47d3a6e3db81e9c69c02" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } health_report { overall_health: UNKNOWN } } }
W20260807 00:01:52.220768 2462 tablet.cc:2369] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:01:52.232473 2679 tablet.cc:2369] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:01:52.235034 2215 tablet.cc:2369] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Can't schedule compaction. Clean time has not been advanced past its initial value.
Aug 07 00:01:52 dist-test-slave-pww0 krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060911, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:52.286360 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.277493 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46713 (local address 127.0.0.1:38664)
0807 00:01:52.277858 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.277874 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.277993 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.278246 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.278248 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.278698 (+ 450us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:52.278804 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.278811 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.279652 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.279655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.280264 (+ 609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.280275 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.280728 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.280730 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.280795 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.281413 (+ 618us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:52.281425 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:52.283000 (+ 1575us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:52.284727 (+ 1727us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:52.284741 (+ 14us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:52.284748 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:52.285015 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:52.285256 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:52.285258 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:52.285259 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:52.285335 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:52.285820 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:52.285827 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:52.285961 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.286063 (+ 102us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.286205 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":264,"thread_start_us":132,"threads_started":1}
I20260807 00:01:52.286394 1963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.277599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38664 (local address 127.30.204.126:46713)
0807 00:01:52.277756 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.277760 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.277982 (+ 222us) server_negotiation.cc:407] Connection header received
0807 00:01:52.278067 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.278070 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.278115 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.278213 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:52.278940 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.279532 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.280402 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.280625 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.280708 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.283164 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:52.283167 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:52.283169 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:52.283194 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:52.284596 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:52.285114 (+ 518us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:52.285116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:52.285117 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:52.285190 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:52.285463 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:52.285466 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:52.285468 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:52.285696 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:52.285787 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.286158 (+ 371us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.286280 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 00:01:52.290901 1891 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:38664:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 00:01:52.291754 1891 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:52.297166 2102 tablet_service.cc:1467] Processing CreateTablet for tablet 5b528d303ddf40f79b860405637bdcee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30a9acb739d24836abd9e83c00c03d51]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:01:52.297277 2566 tablet_service.cc:1467] Processing CreateTablet for tablet 5b528d303ddf40f79b860405637bdcee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30a9acb739d24836abd9e83c00c03d51]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:01:52.297191 2348 tablet_service.cc:1467] Processing CreateTablet for tablet 5b528d303ddf40f79b860405637bdcee (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=30a9acb739d24836abd9e83c00c03d51]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:01:52.298040 2566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b528d303ddf40f79b860405637bdcee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.298040 2102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b528d303ddf40f79b860405637bdcee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.298079 2348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5b528d303ddf40f79b860405637bdcee. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.300779 2717 tablet_bootstrap.cc:492] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: Bootstrap starting.
I20260807 00:01:52.301133 2714 tablet_bootstrap.cc:492] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: Bootstrap starting.
I20260807 00:01:52.301501 2715 tablet_bootstrap.cc:492] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: Bootstrap starting.
I20260807 00:01:52.301908 2717 tablet_bootstrap.cc:654] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.302273 2714 tablet_bootstrap.cc:654] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.302510 2715 tablet_bootstrap.cc:654] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.303979 2717 tablet_bootstrap.cc:492] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: No bootstrap required, opened a new log
I20260807 00:01:52.303980 2715 tablet_bootstrap.cc:492] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: No bootstrap required, opened a new log
I20260807 00:01:52.304086 2715 ts_tablet_manager.cc:1397] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:52.304086 2717 ts_tablet_manager.cc:1397] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:52.304576 2717 raft_consensus.cc:348] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.304576 2715 raft_consensus.cc:348] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.304708 2717 raft_consensus.cc:374] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.304716 2715 raft_consensus.cc:374] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.304751 2715 raft_consensus.cc:729] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec76df22e94404bbe72f3e130170b15, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.304751 2717 raft_consensus.cc:729] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.304895 2717 consensus_queue.cc:260] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.304895 2715 consensus_queue.cc:260] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.305195 2715 ts_tablet_manager.cc:1428] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 00:01:52.305218 2727 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.305388 2727 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:52.305613 2723 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.305773 2723 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:52.305822 2717 ts_tablet_manager.cc:1428] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:01:52.306978 2714 tablet_bootstrap.cc:492] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: No bootstrap required, opened a new log
I20260807 00:01:52.307093 2714 ts_tablet_manager.cc:1397] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 00:01:52.307687 2714 raft_consensus.cc:348] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.307817 2714 raft_consensus.cc:374] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.307849 2714 raft_consensus.cc:729] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073070cedeef47d3a6e3db81e9c69c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.307979 2714 consensus_queue.cc:260] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.308269 2714 ts_tablet_manager.cc:1428] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:52.308256 2722 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.308399 2722 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:52.357545 2727 raft_consensus.cc:482] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:52.357729 2727 raft_consensus.cc:504] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.358165 2727 leader_election.cc:288] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:52.358723 2108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b528d303ddf40f79b860405637bdcee" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02" is_pre_election: true
I20260807 00:01:52.358765 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b528d303ddf40f79b860405637bdcee" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15" is_pre_election: true
I20260807 00:01:52.358903 2108 raft_consensus.cc:2454] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 0.
I20260807 00:01:52.358930 2594 raft_consensus.cc:2454] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 0.
I20260807 00:01:52.359269 2262 leader_election.cc:302] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec76df22e94404bbe72f3e130170b15, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:52.359483 2727 raft_consensus.cc:2781] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:52.359560 2727 raft_consensus.cc:482] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:52.359608 2727 raft_consensus.cc:3037] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.360643 2727 raft_consensus.cc:504] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.360998 2727 leader_election.cc:288] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 election: Requested vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:52.361413 2108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b528d303ddf40f79b860405637bdcee" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02"
I20260807 00:01:52.361477 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5b528d303ddf40f79b860405637bdcee" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15"
I20260807 00:01:52.361548 2108 raft_consensus.cc:3037] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.361606 2594 raft_consensus.cc:3037] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.362564 2108 raft_consensus.cc:2454] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 1.
I20260807 00:01:52.362576 2594 raft_consensus.cc:2454] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 1.
I20260807 00:01:52.362879 2262 leader_election.cc:302] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec76df22e94404bbe72f3e130170b15, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:52.363055 2727 raft_consensus.cc:2781] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:52.363278 2727 raft_consensus.cc:686] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [term 1 LEADER]: Becoming Leader. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Running, Role: LEADER
I20260807 00:01:52.363679 2727 consensus_queue.cc:237] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.365971 2756 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: TxnStatusTablet state changed. Reason: New leader e914d5969c77458685dfa4e7f09513f1. Latest consensus state: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.366112 2756 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:01:52.366571 2758 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 00:01:52.366786 2758 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 00:01:52.367162 1893 catalog_manager.cc:5495] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e914d5969c77458685dfa4e7f09513f1 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } health_report { overall_health: UNKNOWN } } }
I20260807 00:01:52.395933 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.391673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37334 (local address 127.30.204.66:35419)
0807 00:01:52.391793 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.391798 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.391881 (+ 83us) server_negotiation.cc:407] Connection header received
0807 00:01:52.392063 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.392065 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.392125 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.392218 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:52.392479 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.393060 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.394310 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.394542 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.394634 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.394959 (+ 325us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.395545 (+ 586us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.395647 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.395749 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.395811 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260807 00:01:52.395977 2749 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.391588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35419 (local address 127.0.0.1:37334)
0807 00:01:52.391780 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.391796 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.391929 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.392250 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.392253 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.392262 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:52.392385 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.392391 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.393162 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.393165 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.394164 (+ 999us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.394176 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.394659 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.394662 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.394755 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.394819 (+ 64us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.395644 (+ 825us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.395660 (+ 16us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.395726 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.395826 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":54}
I20260807 00:01:52.402086 2594 raft_consensus.cc:1261] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Refusing update from remote peer e914d5969c77458685dfa4e7f09513f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:52.402086 2108 raft_consensus.cc:1261] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Refusing update from remote peer e914d5969c77458685dfa4e7f09513f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:52.402662 2727 consensus_queue.cc:1035] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:52.402871 2756 consensus_queue.cc:1035] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:52.406780 2722 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: TxnStatusTablet state changed. Reason: New leader e914d5969c77458685dfa4e7f09513f1. Latest consensus state: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.406932 2722 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:52.406950 2723 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: TxnStatusTablet state changed. Reason: New leader e914d5969c77458685dfa4e7f09513f1. Latest consensus state: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.407078 2723 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:52.410949 2756 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.411098 2756 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:01:52.411289 2762 mvcc.cc:205] Tried to move back new op lower bound from 7315705497191874560 to 7315705497044381696. Current Snapshot: MvccSnapshot[applied={T|T < 7315705497191874560}]
I20260807 00:01:52.411873 2727 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.411985 2727 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P e914d5969c77458685dfa4e7f09513f1: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:01:52.412431 2766 mvcc.cc:205] Tried to move back new op lower bound from 7315705497191874560 to 7315705497044381696. Current Snapshot: MvccSnapshot[applied={T|T < 7315705497191874560}]
I20260807 00:01:52.412883 2722 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.413007 2722 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P 073070cedeef47d3a6e3db81e9c69c02: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:52.414171 2723 tablet_replica.cc:439] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } } }
I20260807 00:01:52.414296 2723 tablet_replica.cc:442] T 5b528d303ddf40f79b860405637bdcee P 3ec76df22e94404bbe72f3e130170b15: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:01:52.425503 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.421637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57358 (local address 127.30.204.65:33965)
0807 00:01:52.421753 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.421759 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.421823 (+ 64us) server_negotiation.cc:407] Connection header received
0807 00:01:52.422010 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.422013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.422075 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.422176 (+ 101us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:52.422424 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.423000 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.423909 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.424188 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.424281 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.424552 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.425074 (+ 522us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.425192 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.425292 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.425354 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:01:52.437541 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.434059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:54215 (local address 127.30.204.66:35419)
0807 00:01:52.434179 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.434184 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.434388 (+ 204us) server_negotiation.cc:407] Connection header received
0807 00:01:52.434559 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.434562 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.434624 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.434696 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:52.434970 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.435545 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.436433 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.436656 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.436753 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.437074 (+ 321us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.437154 (+ 80us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.437212 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.437325 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.437383 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:01:52.437553 2784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.433962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35419 (local address 127.30.204.65:54215)
0807 00:01:52.434301 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.434315 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.434442 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.434720 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.434723 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.434731 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:52.434871 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.434876 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.435656 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.435660 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.436313 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.436323 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.436765 (+ 442us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.436768 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.436869 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.436917 (+ 48us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.437233 (+ 316us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.437240 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.437306 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.437391 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":99,"threads_started":1}
I20260807 00:01:52.445488 2594 raft_consensus.cc:1261] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Refusing update from remote peer 073070cedeef47d3a6e3db81e9c69c02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:52.445818 2378 raft_consensus.cc:1261] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Refusing update from remote peer 073070cedeef47d3a6e3db81e9c69c02: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:52.445989 2722 consensus_queue.cc:1035] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:52.446384 2739 consensus_queue.cc:1035] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:52.473194 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.468701 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:51495 (local address 127.30.204.65:33965)
0807 00:01:52.468821 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.468826 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.469129 (+ 303us) server_negotiation.cc:407] Connection header received
0807 00:01:52.469238 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.469241 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.469294 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.469353 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:52.469704 (+ 351us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.470560 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.471790 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.472126 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.472199 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.472748 (+ 549us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.472826 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.472885 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.472988 (+ 103us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.473047 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:01:52.473202 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.468631 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:33965 (local address 127.30.204.66:51495)
0807 00:01:52.469028 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.469041 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.469151 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.469470 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.469473 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.469483 (+ 10us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:52.469576 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.469581 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.470686 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.470689 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.471648 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.471661 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.472348 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.472352 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.472436 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.472495 (+ 59us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.472909 (+ 414us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.472917 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.472976 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.473066 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":120,"threads_started":1}
I20260807 00:01:52.505039 1893 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:52.505226 1893 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:52.507793 1893 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Sending DeleteTablet for 3 replicas of tablet 4ecfe42b92bb4514829d22e8c0a125c5
I20260807 00:01:52.508766 2348 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ecfe42b92bb4514829d22e8c0a125c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:52 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:37326
I20260807 00:01:52.509522 2566 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ecfe42b92bb4514829d22e8c0a125c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:52 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 00:01:52.509254 2102 tablet_service.cc:1514] Processing DeleteTablet for tablet 4ecfe42b92bb4514829d22e8c0a125c5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:52 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:57346
I20260807 00:01:52.510013 2800 tablet_replica.cc:330] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: stopping tablet replica
I20260807 00:01:52.510442 2800 raft_consensus.cc:2229] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:52.510766 2800 raft_consensus.cc:2258] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:52.511406 2802 tablet_replica.cc:330] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: stopping tablet replica
I20260807 00:01:52.511305 1893 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 00:01:52.511765 2802 raft_consensus.cc:2229] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 1 LEADER]: Raft consensus shutting down.
W20260807 00:01:52.511831 1893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:52.512183 2802 raft_consensus.cc:2258] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:52.514245 2800 ts_tablet_manager.cc:1905] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:52.518090 2800 ts_tablet_manager.cc:1918] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:52.518311 2800 log.cc:1196] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/4ecfe42b92bb4514829d22e8c0a125c5
I20260807 00:01:52.518399 2102 tablet_service.cc:1467] Processing CreateTablet for tablet bf63c03a66b948abb6bc9fac2d24a63d (DEFAULT_TABLE table=test-table [id=1d2f231b551a4090b8f0b199b7eed3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:52.518847 2102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf63c03a66b948abb6bc9fac2d24a63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.519078 2800 ts_tablet_manager.cc:1939] T 4ecfe42b92bb4514829d22e8c0a125c5 P 3ec76df22e94404bbe72f3e130170b15: Deleting consensus metadata
I20260807 00:01:52.521690 2714 tablet_bootstrap.cc:492] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: Bootstrap starting.
I20260807 00:01:52.522015 2566 tablet_service.cc:1467] Processing CreateTablet for tablet bf63c03a66b948abb6bc9fac2d24a63d (DEFAULT_TABLE table=test-table [id=1d2f231b551a4090b8f0b199b7eed3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:52.522305 2566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf63c03a66b948abb6bc9fac2d24a63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.522800 2714 tablet_bootstrap.cc:654] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.524492 2714 tablet_bootstrap.cc:492] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: No bootstrap required, opened a new log
I20260807 00:01:52.524602 2714 ts_tablet_manager.cc:1397] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:52.524945 2715 tablet_bootstrap.cc:492] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: Bootstrap starting.
I20260807 00:01:52.525112 2714 raft_consensus.cc:348] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.525233 2714 raft_consensus.cc:374] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.525262 2714 raft_consensus.cc:729] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073070cedeef47d3a6e3db81e9c69c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.525250 2348 tablet_service.cc:1467] Processing CreateTablet for tablet bf63c03a66b948abb6bc9fac2d24a63d (DEFAULT_TABLE table=test-table [id=1d2f231b551a4090b8f0b199b7eed3f0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:52.525401 2714 consensus_queue.cc:260] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.525575 2348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf63c03a66b948abb6bc9fac2d24a63d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:52.525704 2714 ts_tablet_manager.cc:1428] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:52.525985 2715 tablet_bootstrap.cc:654] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.527280 2715 tablet_bootstrap.cc:492] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: No bootstrap required, opened a new log
I20260807 00:01:52.527370 2715 ts_tablet_manager.cc:1397] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:01:52.527872 2715 raft_consensus.cc:348] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.527993 2715 raft_consensus.cc:374] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.528026 2715 raft_consensus.cc:729] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec76df22e94404bbe72f3e130170b15, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.527994 1868 catalog_manager.cc:4841] TS 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945): tablet 4ecfe42b92bb4514829d22e8c0a125c5 (table test-table [id=6873dbc077f949708aadba003c521b08]) successfully deleted
I20260807 00:01:52.528331 2715 consensus_queue.cc:260] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.530723 2715 ts_tablet_manager.cc:1428] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: Time spent starting tablet: real 0.003s user 0.001s sys 0.000s
I20260807 00:01:52.532574 2717 tablet_bootstrap.cc:492] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: Bootstrap starting.
I20260807 00:01:52.533931 2717 tablet_bootstrap.cc:654] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:52.534570 2802 ts_tablet_manager.cc:1905] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:52.535478 2717 tablet_bootstrap.cc:492] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: No bootstrap required, opened a new log
I20260807 00:01:52.535581 2717 ts_tablet_manager.cc:1397] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:52.536095 2717 raft_consensus.cc:348] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.536247 2717 raft_consensus.cc:374] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:52.536285 2717 raft_consensus.cc:729] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Initialized, Role: FOLLOWER
I20260807 00:01:52.536422 2717 consensus_queue.cc:260] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.536777 2717 ts_tablet_manager.cc:1428] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:52.538324 2799 tablet_replica.cc:330] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: stopping tablet replica
I20260807 00:01:52.538703 2799 raft_consensus.cc:2229] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:52.541729 2802 ts_tablet_manager.cc:1918] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:52.541877 2802 log.cc:1196] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/4ecfe42b92bb4514829d22e8c0a125c5
I20260807 00:01:52.542269 2802 ts_tablet_manager.cc:1939] T 4ecfe42b92bb4514829d22e8c0a125c5 P 073070cedeef47d3a6e3db81e9c69c02: Deleting consensus metadata
I20260807 00:01:52.543587 2799 raft_consensus.cc:2258] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:52.543615 1868 catalog_manager.cc:4841] TS 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965): tablet 4ecfe42b92bb4514829d22e8c0a125c5 (table test-table [id=6873dbc077f949708aadba003c521b08]) successfully deleted
I20260807 00:01:52.544999 2799 ts_tablet_manager.cc:1905] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:52.548312 2799 ts_tablet_manager.cc:1918] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:52.548432 2799 log.cc:1196] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/4ecfe42b92bb4514829d22e8c0a125c5
I20260807 00:01:52.548725 2799 ts_tablet_manager.cc:1939] T 4ecfe42b92bb4514829d22e8c0a125c5 P e914d5969c77458685dfa4e7f09513f1: Deleting consensus metadata
I20260807 00:01:52.550019 1870 catalog_manager.cc:4841] TS e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419): tablet 4ecfe42b92bb4514829d22e8c0a125c5 (table test-table [id=6873dbc077f949708aadba003c521b08]) successfully deleted
I20260807 00:01:52.881760 2723 raft_consensus.cc:482] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:52.881924 2723 raft_consensus.cc:504] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.883074 2723 leader_election.cc:288] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419)
I20260807 00:01:52.887260 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.883500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:40415 (local address 127.30.204.66:35419)
0807 00:01:52.883651 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.883656 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.883915 (+ 259us) server_negotiation.cc:407] Connection header received
0807 00:01:52.884052 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.884055 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.884126 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.884189 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.884502 (+ 313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.885037 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.886258 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.886672 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.886776 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.886863 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.887011 (+ 148us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.887108 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:01:52.887260 2710 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.883515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:34621 (local address 127.30.204.65:33965)
0807 00:01:52.883649 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.883655 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.883973 (+ 318us) server_negotiation.cc:407] Connection header received
0807 00:01:52.884099 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.884101 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.884153 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.884236 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:01:52.884520 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.885021 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.886258 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.886719 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.886849 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.886906 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.887065 (+ 159us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.887147 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:01:52.887281 2809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.883386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35419 (local address 127.30.204.67:40415)
0807 00:01:52.883795 (+ 409us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.883808 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.883935 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.884222 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.884226 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.884244 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.884376 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.884382 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.885165 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.885167 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.886108 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.886128 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.886777 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.886781 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.886870 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.886894 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.886962 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.887045 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":345,"mutex_wait_us":20,"spinlock_wait_cycles":20736,"thread_start_us":106,"threads_started":1}
I20260807 00:01:52.887290 2811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.883392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:33965 (local address 127.30.204.67:34621)
0807 00:01:52.883858 (+ 466us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.883871 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.883995 (+ 124us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.884256 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.884259 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.884308 (+ 49us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:01:52.884407 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.884412 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.885137 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.885141 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.886115 (+ 974us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.886127 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.886847 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.886851 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.886943 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.886966 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.887038 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.887110 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":408,"mutex_wait_us":151,"spinlock_wait_cycles":2688,"thread_start_us":87,"threads_started":1}
I20260807 00:01:52.887964 2378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf63c03a66b948abb6bc9fac2d24a63d" candidate_uuid: "3ec76df22e94404bbe72f3e130170b15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e914d5969c77458685dfa4e7f09513f1" is_pre_election: true
I20260807 00:01:52.888127 2378 raft_consensus.cc:2454] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ec76df22e94404bbe72f3e130170b15 in term 0.
I20260807 00:01:52.888231 2108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf63c03a66b948abb6bc9fac2d24a63d" candidate_uuid: "3ec76df22e94404bbe72f3e130170b15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02" is_pre_election: true
I20260807 00:01:52.888388 2108 raft_consensus.cc:2454] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 3ec76df22e94404bbe72f3e130170b15 in term 0.
I20260807 00:01:52.888552 2501 leader_election.cc:302] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec76df22e94404bbe72f3e130170b15, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:52.889003 2723 raft_consensus.cc:2781] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:52.889055 2723 raft_consensus.cc:482] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:52.889079 2723 raft_consensus.cc:3037] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.890154 2723 raft_consensus.cc:504] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.890555 2723 leader_election.cc:288] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [CANDIDATE]: Term 1 election: Requested vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419)
I20260807 00:01:52.890971 2108 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf63c03a66b948abb6bc9fac2d24a63d" candidate_uuid: "3ec76df22e94404bbe72f3e130170b15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02"
I20260807 00:01:52.890971 2378 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf63c03a66b948abb6bc9fac2d24a63d" candidate_uuid: "3ec76df22e94404bbe72f3e130170b15" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e914d5969c77458685dfa4e7f09513f1"
I20260807 00:01:52.891117 2378 raft_consensus.cc:3037] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.891116 2108 raft_consensus.cc:3037] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:52.892393 2378 raft_consensus.cc:2454] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ec76df22e94404bbe72f3e130170b15 in term 1.
I20260807 00:01:52.892393 2108 raft_consensus.cc:2454] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 3ec76df22e94404bbe72f3e130170b15 in term 1.
I20260807 00:01:52.892791 2499 leader_election.cc:302] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073070cedeef47d3a6e3db81e9c69c02, 3ec76df22e94404bbe72f3e130170b15; no voters:
I20260807 00:01:52.893038 2723 raft_consensus.cc:2781] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:52.893193 2723 raft_consensus.cc:686] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 1 LEADER]: Becoming Leader. State: Replica: 3ec76df22e94404bbe72f3e130170b15, State: Running, Role: LEADER
I20260807 00:01:52.893472 2723 consensus_queue.cc:237] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:52.895898 1893 catalog_manager.cc:5495] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 reported cstate change: term changed from 0 to 1, leader changed from <none> to 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67). New cstate: current_term: 1 leader_uuid: "3ec76df22e94404bbe72f3e130170b15" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } health_report { overall_health: HEALTHY } } }
I20260807 00:01:52.916456 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.912396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46126 (local address 127.30.204.67:33945)
0807 00:01:52.912684 (+ 288us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.912690 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.912705 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:01:52.912784 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.912786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.912843 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.912945 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:52.913201 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.913746 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.914607 (+ 861us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.914855 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.914937 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.915276 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.915875 (+ 599us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.916000 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.916092 (+ 92us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.916150 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":221,"thread_start_us":130,"threads_started":1}
I20260807 00:01:52.928143 2704 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.924548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:55889 (local address 127.30.204.66:35419)
0807 00:01:52.924687 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.924691 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.924816 (+ 125us) server_negotiation.cc:407] Connection header received
0807 00:01:52.924951 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.924953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.925001 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.925069 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:52.925356 (+ 287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.925885 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.926887 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.927142 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.927237 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.927673 (+ 436us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.927780 (+ 107us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.927843 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.927956 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.928012 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:01:52.928174 2826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.924434 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35419 (local address 127.30.204.67:55889)
0807 00:01:52.924742 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.924755 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.924859 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.925084 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.925086 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.925094 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:52.925270 (+ 176us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.925276 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.926027 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.926031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.926742 (+ 711us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.926754 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.927328 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.927331 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.927412 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.927517 (+ 105us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.927865 (+ 348us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.927872 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.927938 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.928020 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
I20260807 00:01:52.934314 2108 raft_consensus.cc:1261] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Refusing update from remote peer 3ec76df22e94404bbe72f3e130170b15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:52.934314 2378 raft_consensus.cc:1261] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Refusing update from remote peer 3ec76df22e94404bbe72f3e130170b15: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:52.934783 2723 consensus_queue.cc:1035] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:52.934985 2812 consensus_queue.cc:1035] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:52.952478 2820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.949310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:49539 (local address 127.30.204.67:33945)
0807 00:01:52.949423 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:52.949426 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:52.949440 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:01:52.949587 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:52.949590 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:52.949645 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:52.949730 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:52.949909 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.950559 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.951538 (+ 979us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.951789 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.951873 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.952065 (+ 192us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.952135 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.952186 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:52.952288 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 00:01:52.952336 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:01:52.952498 2792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:52.949226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:33945 (local address 127.30.204.66:49539)
0807 00:01:52.949367 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:52.949382 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:52.949468 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:52.949735 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:52.949737 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:52.949743 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:52.949834 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.949838 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.950691 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.950694 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.951394 (+ 700us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:52.951405 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:52.951885 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:52.951888 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:52.951950 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:52.951979 (+ 29us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:52.952204 (+ 225us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:52.952210 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:01:52.952281 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:01:52.952383 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 00:01:52.953394 2766 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 00:01:54.520123 1893 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:54.520293 1893 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38650:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:54.523093 1893 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Sending DeleteTablet for 3 replicas of tablet bf63c03a66b948abb6bc9fac2d24a63d
I20260807 00:01:54.523880 2102 tablet_service.cc:1514] Processing DeleteTablet for tablet bf63c03a66b948abb6bc9fac2d24a63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:54 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:57346
I20260807 00:01:54.524147 2566 tablet_service.cc:1514] Processing DeleteTablet for tablet bf63c03a66b948abb6bc9fac2d24a63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:54 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 00:01:54.524289 2348 tablet_service.cc:1514] Processing DeleteTablet for tablet bf63c03a66b948abb6bc9fac2d24a63d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:54 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:37326
I20260807 00:01:54.525372 2872 tablet_replica.cc:330] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: stopping tablet replica
I20260807 00:01:54.525529 2872 raft_consensus.cc:2229] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:54.525674 2872 raft_consensus.cc:2258] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:54.526401 2872 ts_tablet_manager.cc:1905] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:54.527577 2873 tablet_replica.cc:330] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: stopping tablet replica
I20260807 00:01:54.527777 2873 raft_consensus.cc:2229] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:01:54.528141 2873 raft_consensus.cc:2258] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:54.528422 2874 tablet_replica.cc:330] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: stopping tablet replica
I20260807 00:01:54.528563 2874 raft_consensus.cc:2229] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:54.528715 2874 raft_consensus.cc:2258] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:54.528765 2873 ts_tablet_manager.cc:1905] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:54.529343 2874 ts_tablet_manager.cc:1905] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:54.529456 2872 ts_tablet_manager.cc:1918] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:01:54.529544 2872 log.cc:1196] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/bf63c03a66b948abb6bc9fac2d24a63d
I20260807 00:01:54.529804 2872 ts_tablet_manager.cc:1939] T bf63c03a66b948abb6bc9fac2d24a63d P 073070cedeef47d3a6e3db81e9c69c02: Deleting consensus metadata
I20260807 00:01:54.530728 1868 catalog_manager.cc:4841] TS 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965): tablet bf63c03a66b948abb6bc9fac2d24a63d (table test-table [id=1d2f231b551a4090b8f0b199b7eed3f0]) successfully deleted
I20260807 00:01:54.531750 2874 ts_tablet_manager.cc:1918] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:01:54.531752 2873 ts_tablet_manager.cc:1918] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:01:54.531823 2873 log.cc:1196] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/bf63c03a66b948abb6bc9fac2d24a63d
I20260807 00:01:54.531826 2874 log.cc:1196] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/bf63c03a66b948abb6bc9fac2d24a63d
I20260807 00:01:54.532096 2873 ts_tablet_manager.cc:1939] T bf63c03a66b948abb6bc9fac2d24a63d P 3ec76df22e94404bbe72f3e130170b15: Deleting consensus metadata
I20260807 00:01:54.532112 2874 ts_tablet_manager.cc:1939] T bf63c03a66b948abb6bc9fac2d24a63d P e914d5969c77458685dfa4e7f09513f1: Deleting consensus metadata
I20260807 00:01:54.532837 1870 catalog_manager.cc:4841] TS e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419): tablet bf63c03a66b948abb6bc9fac2d24a63d (table test-table [id=1d2f231b551a4090b8f0b199b7eed3f0]) successfully deleted
I20260807 00:01:54.532838 1868 catalog_manager.cc:4841] TS 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945): tablet bf63c03a66b948abb6bc9fac2d24a63d (table test-table [id=1d2f231b551a4090b8f0b199b7eed3f0]) successfully deleted
Aug 07 00:01:54 dist-test-slave-pww0 krb5kdc[1822](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:01:54 dist-test-slave-pww0 krb5kdc[1822](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060914, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:54.562135 2889 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:54.547607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.30.204.126:46713)
0807 00:01:54.547887 (+ 280us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:54.547895 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:54.547946 (+ 51us) server_negotiation.cc:407] Connection header received
0807 00:01:54.548082 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:54.548085 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:54.548153 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:54.548249 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:54.549036 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.549632 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.550522 (+ 890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.550801 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.550900 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:54.559006 (+ 8106us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:54.559010 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:54.559012 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:54.559052 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:54.560490 (+ 1438us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:54.560970 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:54.560973 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:54.560975 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:54.561064 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:54.561277 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:54.561279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:54.561280 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:54.561481 (+ 201us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:54.561582 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:54.561839 (+ 257us) server_negotiation.cc:299] Negotiation successful
0807 00:01:54.561956 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":105,"threads_started":1}
I20260807 00:01:54.564998 1893 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:01:54.565474 1893 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:54.570760 2102 tablet_service.cc:1467] Processing CreateTablet for tablet 89402e3c5a834e6790d625af89b87c6a (DEFAULT_TABLE table=test-table [id=a4cfd5e2931342a695e73f55a17efc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:54.570853 2348 tablet_service.cc:1467] Processing CreateTablet for tablet 89402e3c5a834e6790d625af89b87c6a (DEFAULT_TABLE table=test-table [id=a4cfd5e2931342a695e73f55a17efc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:54.571136 2348 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89402e3c5a834e6790d625af89b87c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:54.571141 2102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89402e3c5a834e6790d625af89b87c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:54.571200 2566 tablet_service.cc:1467] Processing CreateTablet for tablet 89402e3c5a834e6790d625af89b87c6a (DEFAULT_TABLE table=test-table [id=a4cfd5e2931342a695e73f55a17efc28]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:54.571537 2566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 89402e3c5a834e6790d625af89b87c6a. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:54.574131 2893 tablet_bootstrap.cc:492] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: Bootstrap starting.
I20260807 00:01:54.574168 2892 tablet_bootstrap.cc:492] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: Bootstrap starting.
I20260807 00:01:54.574304 2891 tablet_bootstrap.cc:492] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: Bootstrap starting.
I20260807 00:01:54.574966 2893 tablet_bootstrap.cc:654] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:54.575098 2892 tablet_bootstrap.cc:654] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:54.575106 2891 tablet_bootstrap.cc:654] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:54.576282 2893 tablet_bootstrap.cc:492] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: No bootstrap required, opened a new log
I20260807 00:01:54.576282 2892 tablet_bootstrap.cc:492] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: No bootstrap required, opened a new log
I20260807 00:01:54.576375 2893 ts_tablet_manager.cc:1397] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260807 00:01:54.576378 2892 ts_tablet_manager.cc:1397] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:01:54.576785 2892 raft_consensus.cc:348] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.576803 2893 raft_consensus.cc:348] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.576915 2893 raft_consensus.cc:374] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:54.576917 2892 raft_consensus.cc:374] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:54.576947 2893 raft_consensus.cc:729] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec76df22e94404bbe72f3e130170b15, State: Initialized, Role: FOLLOWER
I20260807 00:01:54.576957 2892 raft_consensus.cc:729] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073070cedeef47d3a6e3db81e9c69c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:54.577080 2893 consensus_queue.cc:260] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.577118 2892 consensus_queue.cc:260] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.577359 2893 ts_tablet_manager.cc:1428] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: Time spent starting tablet: real 0.001s user 0.000s sys 0.001s
I20260807 00:01:54.577483 2892 ts_tablet_manager.cc:1428] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:54.578004 2891 tablet_bootstrap.cc:492] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: No bootstrap required, opened a new log
I20260807 00:01:54.578109 2891 ts_tablet_manager.cc:1397] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260807 00:01:54.578589 2891 raft_consensus.cc:348] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.578718 2891 raft_consensus.cc:374] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:54.578751 2891 raft_consensus.cc:729] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Initialized, Role: FOLLOWER
I20260807 00:01:54.578925 2891 consensus_queue.cc:260] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.579219 2891 ts_tablet_manager.cc:1428] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:54.883095 2839 raft_consensus.cc:482] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:54.883260 2839 raft_consensus.cc:504] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.883744 2839 leader_election.cc:288] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:54.884328 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89402e3c5a834e6790d625af89b87c6a" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15" is_pre_election: true
I20260807 00:01:54.884341 2136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89402e3c5a834e6790d625af89b87c6a" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02" is_pre_election: true
I20260807 00:01:54.884502 2594 raft_consensus.cc:2454] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 0.
I20260807 00:01:54.884511 2136 raft_consensus.cc:2454] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 0.
I20260807 00:01:54.884879 2262 leader_election.cc:302] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec76df22e94404bbe72f3e130170b15, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:54.885076 2839 raft_consensus.cc:2781] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:54.885142 2839 raft_consensus.cc:482] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:54.885190 2839 raft_consensus.cc:3037] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:54.886291 2839 raft_consensus.cc:504] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.886656 2839 leader_election.cc:288] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 election: Requested vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:54.887125 2136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89402e3c5a834e6790d625af89b87c6a" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02"
I20260807 00:01:54.887122 2594 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "89402e3c5a834e6790d625af89b87c6a" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15"
I20260807 00:01:54.887265 2594 raft_consensus.cc:3037] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:54.887265 2136 raft_consensus.cc:3037] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:54.888554 2136 raft_consensus.cc:2454] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 1.
I20260807 00:01:54.888554 2594 raft_consensus.cc:2454] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 1.
I20260807 00:01:54.888904 2262 leader_election.cc:302] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec76df22e94404bbe72f3e130170b15, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:54.889081 2839 raft_consensus.cc:2781] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:54.889205 2839 raft_consensus.cc:686] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 1 LEADER]: Becoming Leader. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Running, Role: LEADER
I20260807 00:01:54.889366 2839 consensus_queue.cc:237] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:54.891259 1888 catalog_manager.cc:5495] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e914d5969c77458685dfa4e7f09513f1 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } health_report { overall_health: UNKNOWN } } }
I20260807 00:01:54.952608 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:54.949230 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37340 (local address 127.30.204.66:35419)
0807 00:01:54.949548 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:54.949556 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:54.949567 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:01:54.949610 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:54.949612 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:54.949654 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:54.949726 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:54.950030 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.950581 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.951489 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.951725 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.951823 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:54.952108 (+ 285us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:54.952198 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:54.952266 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:54.952367 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 00:01:54.952434 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":243,"thread_start_us":116,"threads_started":1}
I20260807 00:01:54.963282 2906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:54.960339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:48069 (local address 127.30.204.66:35419)
0807 00:01:54.960448 (+ 109us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:54.960451 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:54.960524 (+ 73us) server_negotiation.cc:407] Connection header received
0807 00:01:54.960635 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:54.960637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:54.960683 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:54.960752 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:01:54.960926 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.961428 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.962309 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.962533 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.962655 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:54.962888 (+ 233us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:01:54.962961 (+ 73us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:01:54.963009 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:54.963099 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 00:01:54.963151 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":1,"server-negotiator.queue_time_us":40}
I20260807 00:01:54.963294 2909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:54.960112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35419 (local address 127.30.204.66:48069)
0807 00:01:54.960435 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:54.960448 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:54.960553 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:54.960765 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:54.960767 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:54.960773 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:01:54.960864 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:54.960867 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.961543 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.961546 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:54.962176 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:54.962185 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:54.962676 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:54.962678 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:54.962767 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:54.962797 (+ 30us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:01:54.963033 (+ 236us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:01:54.963039 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:01:54.963085 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 00:01:54.963162 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":233,"thread_start_us":125,"threads_started":1}
I20260807 00:01:54.969683 2594 raft_consensus.cc:1261] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Refusing update from remote peer e914d5969c77458685dfa4e7f09513f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:54.969683 2136 raft_consensus.cc:1261] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Refusing update from remote peer e914d5969c77458685dfa4e7f09513f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:54.970139 2849 consensus_queue.cc:1035] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:54.970351 2843 consensus_queue.cc:1035] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:54.975526 2844 mvcc.cc:205] Tried to move back new op lower bound from 7315705507712434176 to 7315705507388526592. Current Snapshot: MvccSnapshot[applied={T|T < 7315705507712434176}]
I20260807 00:01:55.011168 1888 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:55.011374 1888 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:55.013880 1888 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Sending DeleteTablet for 3 replicas of tablet 89402e3c5a834e6790d625af89b87c6a
I20260807 00:01:55.014681 2347 tablet_service.cc:1514] Processing DeleteTablet for tablet 89402e3c5a834e6790d625af89b87c6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:55 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:37326
I20260807 00:01:55.014767 2566 tablet_service.cc:1514] Processing DeleteTablet for tablet 89402e3c5a834e6790d625af89b87c6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:55 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 00:01:55.014925 2874 tablet_replica.cc:330] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: stopping tablet replica
I20260807 00:01:55.014972 2102 tablet_service.cc:1514] Processing DeleteTablet for tablet 89402e3c5a834e6790d625af89b87c6a with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:55 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:57346
I20260807 00:01:55.015057 2874 raft_consensus.cc:2229] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:01:55.015187 2873 tablet_replica.cc:330] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: stopping tablet replica
I20260807 00:01:55.015192 2872 tablet_replica.cc:330] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: stopping tablet replica
I20260807 00:01:55.015296 2873 raft_consensus.cc:2229] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:55.015333 2872 raft_consensus.cc:2229] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:55.015400 2873 raft_consensus.cc:2258] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:55.015386 2874 raft_consensus.cc:2258] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:55.015482 2872 raft_consensus.cc:2258] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:55.016295 2872 ts_tablet_manager.cc:1905] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:55.016294 2873 ts_tablet_manager.cc:1905] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:55.017249 1888 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38678:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:01:55.017746 1888 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:01:55.019156 2873 ts_tablet_manager.cc:1918] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:55.019157 2872 ts_tablet_manager.cc:1918] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:55.019232 2873 log.cc:1196] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/89402e3c5a834e6790d625af89b87c6a
I20260807 00:01:55.019236 2872 log.cc:1196] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/89402e3c5a834e6790d625af89b87c6a
I20260807 00:01:55.019579 2872 ts_tablet_manager.cc:1939] T 89402e3c5a834e6790d625af89b87c6a P 073070cedeef47d3a6e3db81e9c69c02: Deleting consensus metadata
I20260807 00:01:55.019785 2873 ts_tablet_manager.cc:1939] T 89402e3c5a834e6790d625af89b87c6a P 3ec76df22e94404bbe72f3e130170b15: Deleting consensus metadata
I20260807 00:01:55.020444 1868 catalog_manager.cc:4841] TS 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965): tablet 89402e3c5a834e6790d625af89b87c6a (table test-table [id=a4cfd5e2931342a695e73f55a17efc28]) successfully deleted
I20260807 00:01:55.020962 1868 catalog_manager.cc:4841] TS 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945): tablet 89402e3c5a834e6790d625af89b87c6a (table test-table [id=a4cfd5e2931342a695e73f55a17efc28]) successfully deleted
I20260807 00:01:55.021945 2874 ts_tablet_manager.cc:1905] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:55.024397 2874 ts_tablet_manager.cc:1918] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:01:55.024499 2874 log.cc:1196] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/89402e3c5a834e6790d625af89b87c6a
I20260807 00:01:55.025395 2102 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7856956e644b97be01edd4122d517d (DEFAULT_TABLE table=test-table [id=69ac034787934b73ba80a91a1c57fe18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:55.025619 2566 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7856956e644b97be01edd4122d517d (DEFAULT_TABLE table=test-table [id=69ac034787934b73ba80a91a1c57fe18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:55.025734 2102 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f7856956e644b97be01edd4122d517d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:55.025875 2566 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f7856956e644b97be01edd4122d517d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:55.028210 2892 tablet_bootstrap.cc:492] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02: Bootstrap starting.
I20260807 00:01:55.029172 2892 tablet_bootstrap.cc:654] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:55.029313 2874 ts_tablet_manager.cc:1939] T 89402e3c5a834e6790d625af89b87c6a P e914d5969c77458685dfa4e7f09513f1: Deleting consensus metadata
I20260807 00:01:55.029812 2893 tablet_bootstrap.cc:492] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: Bootstrap starting.
I20260807 00:01:55.030058 1870 catalog_manager.cc:4841] TS e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419): tablet 89402e3c5a834e6790d625af89b87c6a (table test-table [id=a4cfd5e2931342a695e73f55a17efc28]) successfully deleted
I20260807 00:01:55.028492 2347 tablet_service.cc:1467] Processing CreateTablet for tablet 0f7856956e644b97be01edd4122d517d (DEFAULT_TABLE table=test-table [id=69ac034787934b73ba80a91a1c57fe18]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:01:55.030387 2347 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0f7856956e644b97be01edd4122d517d. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:55.030674 2893 tablet_bootstrap.cc:654] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:55.031767 2892 tablet_bootstrap.cc:492] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02: No bootstrap required, opened a new log
I20260807 00:01:55.031875 2892 ts_tablet_manager.cc:1397] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 00:01:55.032388 2892 raft_consensus.cc:348] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.032533 2892 raft_consensus.cc:374] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:55.032564 2892 raft_consensus.cc:729] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 073070cedeef47d3a6e3db81e9c69c02, State: Initialized, Role: FOLLOWER
I20260807 00:01:55.032612 2891 tablet_bootstrap.cc:492] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: Bootstrap starting.
I20260807 00:01:55.032836 2892 consensus_queue.cc:260] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.033468 2891 tablet_bootstrap.cc:654] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:55.034480 2891 tablet_bootstrap.cc:492] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: No bootstrap required, opened a new log
I20260807 00:01:55.034576 2891 ts_tablet_manager.cc:1397] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:01:55.034950 2891 raft_consensus.cc:348] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.035050 2891 raft_consensus.cc:374] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:55.035082 2891 raft_consensus.cc:729] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Initialized, Role: FOLLOWER
I20260807 00:01:55.035198 2891 consensus_queue.cc:260] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.035293 2893 tablet_bootstrap.cc:492] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: No bootstrap required, opened a new log
I20260807 00:01:55.035384 2893 ts_tablet_manager.cc:1397] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260807 00:01:55.035482 2891 ts_tablet_manager.cc:1428] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:55.035820 2893 raft_consensus.cc:348] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.035941 2893 raft_consensus.cc:374] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:55.035981 2893 raft_consensus.cc:729] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ec76df22e94404bbe72f3e130170b15, State: Initialized, Role: FOLLOWER
I20260807 00:01:55.036109 2893 consensus_queue.cc:260] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.036388 2893 ts_tablet_manager.cc:1428] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:01:55.037995 2892 ts_tablet_manager.cc:1428] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02: Time spent starting tablet: real 0.006s user 0.000s sys 0.001s
I20260807 00:01:55.063506 2843 raft_consensus.cc:482] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:01:55.063663 2843 raft_consensus.cc:504] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.064122 2843 leader_election.cc:288] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:55.064744 2136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7856956e644b97be01edd4122d517d" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02" is_pre_election: true
I20260807 00:01:55.064742 2598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7856956e644b97be01edd4122d517d" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15" is_pre_election: true
I20260807 00:01:55.064913 2136 raft_consensus.cc:2454] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 0.
I20260807 00:01:55.064920 2598 raft_consensus.cc:2454] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 0.
I20260807 00:01:55.065284 2262 leader_election.cc:302] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3ec76df22e94404bbe72f3e130170b15, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:55.065483 2843 raft_consensus.cc:2781] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:01:55.065547 2843 raft_consensus.cc:482] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:01:55.065593 2843 raft_consensus.cc:3037] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:55.066844 2843 raft_consensus.cc:504] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.067243 2843 leader_election.cc:288] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 election: Requested vote from peers 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965), 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945)
I20260807 00:01:55.067682 2136 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7856956e644b97be01edd4122d517d" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "073070cedeef47d3a6e3db81e9c69c02"
I20260807 00:01:55.067721 2598 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0f7856956e644b97be01edd4122d517d" candidate_uuid: "e914d5969c77458685dfa4e7f09513f1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3ec76df22e94404bbe72f3e130170b15"
I20260807 00:01:55.067840 2136 raft_consensus.cc:3037] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:55.067840 2598 raft_consensus.cc:3037] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:55.069103 2136 raft_consensus.cc:2454] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 1.
I20260807 00:01:55.069103 2598 raft_consensus.cc:2454] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e914d5969c77458685dfa4e7f09513f1 in term 1.
I20260807 00:01:55.069509 2262 leader_election.cc:302] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 073070cedeef47d3a6e3db81e9c69c02, e914d5969c77458685dfa4e7f09513f1; no voters:
I20260807 00:01:55.069869 2843 raft_consensus.cc:2781] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:55.069994 2843 raft_consensus.cc:686] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 1 LEADER]: Becoming Leader. State: Replica: e914d5969c77458685dfa4e7f09513f1, State: Running, Role: LEADER
I20260807 00:01:55.070142 2843 consensus_queue.cc:237] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } }
I20260807 00:01:55.072278 1893 catalog_manager.cc:5495] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 reported cstate change: term changed from 0 to 1, leader changed from <none> to e914d5969c77458685dfa4e7f09513f1 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "e914d5969c77458685dfa4e7f09513f1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e914d5969c77458685dfa4e7f09513f1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35419 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 } health_report { overall_health: UNKNOWN } } }
I20260807 00:01:55.092772 2598 raft_consensus.cc:1261] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Refusing update from remote peer e914d5969c77458685dfa4e7f09513f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:55.092772 2136 raft_consensus.cc:1261] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02 [term 1 FOLLOWER]: Refusing update from remote peer e914d5969c77458685dfa4e7f09513f1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:01:55.093202 2839 consensus_queue.cc:1035] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "073070cedeef47d3a6e3db81e9c69c02" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 33965 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:01:55.093545 2913 consensus_queue.cc:1035] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3ec76df22e94404bbe72f3e130170b15" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 33945 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 00:01:55.110801 2840 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260807 00:01:56.680212 1893 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:56.680378 1893 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:38678:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:01:56.683187 1893 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4f9f34a7db374e3eb11059ecd683804f: Sending DeleteTablet for 3 replicas of tablet 0f7856956e644b97be01edd4122d517d
I20260807 00:01:56.683944 2102 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f7856956e644b97be01edd4122d517d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:56 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:57346
I20260807 00:01:56.683998 2347 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f7856956e644b97be01edd4122d517d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:56 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:37326
I20260807 00:01:56.683990 2566 tablet_service.cc:1514] Processing DeleteTablet for tablet 0f7856956e644b97be01edd4122d517d with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:01:56 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:46122
I20260807 00:01:56.684376 2950 tablet_replica.cc:330] T 0f7856956e644b97be01edd4122d517d P 073070cedeef47d3a6e3db81e9c69c02: stopping tablet replica
I20260807 00:01:56.684382 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2002
I20260807 00:01:56.699501 2951 tablet_replica.cc:330] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: stopping tablet replica
I20260807 00:01:56.699692 2951 raft_consensus.cc:2229] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:01:56.699734 2952 tablet_replica.cc:330] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: stopping tablet replica
I20260807 00:01:56.699918 2952 raft_consensus.cc:2229] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:01:56.699955 2951 raft_consensus.cc:2258] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:56.700063 2952 raft_consensus.cc:2258] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:01:56.700810 2951 ts_tablet_manager.cc:1905] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:56.700809 2952 ts_tablet_manager.cc:1905] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:01:56.703760 2952 ts_tablet_manager.cc:1918] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:01:56.703760 2951 ts_tablet_manager.cc:1918] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260807 00:01:56.703856 2951 log.cc:1196] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/0f7856956e644b97be01edd4122d517d
I20260807 00:01:56.703909 2952 log.cc:1196] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/0f7856956e644b97be01edd4122d517d
I20260807 00:01:56.704195 2951 ts_tablet_manager.cc:1939] T 0f7856956e644b97be01edd4122d517d P e914d5969c77458685dfa4e7f09513f1: Deleting consensus metadata
I20260807 00:01:56.704195 2952 ts_tablet_manager.cc:1939] T 0f7856956e644b97be01edd4122d517d P 3ec76df22e94404bbe72f3e130170b15: Deleting consensus metadata
W20260807 00:01:56.704133 1868 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:33965 (error 108)
I20260807 00:01:56.704977 1870 catalog_manager.cc:4841] TS e914d5969c77458685dfa4e7f09513f1 (127.30.204.66:35419): tablet 0f7856956e644b97be01edd4122d517d (table test-table [id=69ac034787934b73ba80a91a1c57fe18]) successfully deleted
I20260807 00:01:56.705215 1868 catalog_manager.cc:4841] TS 3ec76df22e94404bbe72f3e130170b15 (127.30.204.67:33945): tablet 0f7856956e644b97be01edd4122d517d (table test-table [id=69ac034787934b73ba80a91a1c57fe18]) successfully deleted
I20260807 00:01:56.705425 2955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:56.704973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:33965 (local address 127.0.0.1:57364)
0807 00:01:56.705241 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:56.705318 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:33965: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":88,"threads_started":1}
W20260807 00:01:56.705628 1868 catalog_manager.cc:4584] TS 073070cedeef47d3a6e3db81e9c69c02 (127.30.204.65:33965): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0f7856956e644b97be01edd4122d517d: Network error: Client connection negotiation failed: client connection to 127.30.204.65:33965: connect: Connection refused (error 111)
I20260807 00:01:56.705765 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2238
I20260807 00:01:56.725219 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2472
I20260807 00:01:56.740000 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1841
2026-08-07T00:01:56Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (7121 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:01:56 dist-test-slave-pww0 krb5kdc[2961](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:01:56 dist-test-slave-pww0 krb5kdc[2961](info): set up 2 sockets
Aug 07 00:01:56 dist-test-slave-pww0 krb5kdc[2961](info): commencing operation
krb5kdc: starting...
W20260807 00:01:57.898687 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.128s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:01:57 dist-test-slave-pww0 krb5kdc[2961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:01:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:01:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:01:58.031436 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39247
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:39247
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:58.150398 2979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:58.150890 2979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:58.150991 2979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:58.155544 2979 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:01:58.155658 2979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:58.155691 2979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:58.155714 2979 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:01:58.155736 2979 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:01:58.161823 2979 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:39247
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39247
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:58.163070 2979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:58.164258 2979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:58.173760 2989 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:58.173771 2993 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:58.173738 2987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:58.173938 2979 server_base.cc:1034] running on GCE node
I20260807 00:01:58.174796 2979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:58.176178 2979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:58.177372 2979 hybrid_clock.cc:648] HybridClock initialized: now 1786060918177356 us; error 41 us; skew 500 ppm
Aug 07 00:01:58 dist-test-slave-pww0 krb5kdc[2961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:58.180490 2979 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:01:58.181514 2979 webserver.cc:460] Webserver started at http://127.30.204.126:35199/ using document root <none> and password file <none>
I20260807 00:01:58.182109 2979 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:58.182193 2979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:58.182417 2979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:58.184723 2979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "1dd4c9d8142846deb019464b672ac565"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "9888b792f35da2b3d2fabd7ed3a20a64"
server_key_iv: "5e571b954e2099d85429c8d55efe4240"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.185352 2979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "1dd4c9d8142846deb019464b672ac565"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "9888b792f35da2b3d2fabd7ed3a20a64"
server_key_iv: "5e571b954e2099d85429c8d55efe4240"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.189653 2979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:01:58.192544 3007 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.194010 2979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:58.194166 2979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "1dd4c9d8142846deb019464b672ac565"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "9888b792f35da2b3d2fabd7ed3a20a64"
server_key_iv: "5e571b954e2099d85429c8d55efe4240"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.194303 2979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:58.231854 2979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:58.235718 2979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:58.235949 2979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:58.243525 2979 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:39247
I20260807 00:01:58.243569 3081 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:39247 every 8 connection(s)
I20260807 00:01:58.244617 2979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:01:58.247879 3083 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:01:58.250651 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2979
I20260807 00:01:58.250769 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:01:58.251024 31537 external_mini_cluster.cc:1428] Setting key b2a29db8d9778899f8d09754f988204e
I20260807 00:01:58.254586 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565: Bootstrap starting.
I20260807 00:01:58.257108 3083 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565: Neither blocks nor log segments found. Creating new log.
I20260807 00:01:58.257921 3083 log.cc:824] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565: Log is configured to *not* fsync() on all Append() calls
Aug 07 00:01:58 dist-test-slave-pww0 krb5kdc[2961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060917, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:58.260589 3083 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565: No bootstrap required, opened a new log
I20260807 00:01:58.265515 3083 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd4c9d8142846deb019464b672ac565" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39247 } }
I20260807 00:01:58.265925 3083 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:01:58.266009 3083 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1dd4c9d8142846deb019464b672ac565, State: Initialized, Role: FOLLOWER
I20260807 00:01:58.266582 3083 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd4c9d8142846deb019464b672ac565" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39247 } }
I20260807 00:01:58.266758 3083 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:01:58.266829 3083 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:01:58.266935 3083 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:01:58.267241 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:58.252658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33024 (local address 127.30.204.126:39247)
0807 00:01:58.253235 (+ 577us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:58.253254 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:58.253298 (+ 44us) server_negotiation.cc:407] Connection header received
0807 00:01:58.254017 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:58.254037 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:58.254415 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:58.254774 (+ 359us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:58.255712 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.256761 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.257711 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.258239 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.258403 (+ 164us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:58.261210 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:58.261216 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:58.261236 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:58.261274 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:58.263514 (+ 2240us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:58.264017 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:58.264023 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:58.264030 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:58.264208 (+ 178us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:58.264569 (+ 361us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:58.264573 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:58.264574 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:58.265208 (+ 634us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:58.265429 (+ 221us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:58.265730 (+ 301us) server_negotiation.cc:299] Negotiation successful
0807 00:01:58.266098 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":164,"threads_started":1}
I20260807 00:01:58.268313 3083 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd4c9d8142846deb019464b672ac565" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39247 } }
I20260807 00:01:58.268714 3083 leader_election.cc:302] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1dd4c9d8142846deb019464b672ac565; no voters:
I20260807 00:01:58.269068 3083 leader_election.cc:288] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:01:58.269191 3090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:01:58.269462 3090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [term 1 LEADER]: Becoming Leader. State: Replica: 1dd4c9d8142846deb019464b672ac565, State: Running, Role: LEADER
I20260807 00:01:58.269879 3090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd4c9d8142846deb019464b672ac565" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39247 } }
I20260807 00:01:58.270500 3083 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:01:58.271235 3090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1dd4c9d8142846deb019464b672ac565. Latest consensus state: current_term: 1 leader_uuid: "1dd4c9d8142846deb019464b672ac565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd4c9d8142846deb019464b672ac565" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39247 } } }
I20260807 00:01:58.271499 3090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:58.271790 3093 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1dd4c9d8142846deb019464b672ac565" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1dd4c9d8142846deb019464b672ac565" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39247 } } }
I20260807 00:01:58.271973 3093 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565 [sys.catalog]: This master's current role is: LEADER
I20260807 00:01:58.271991 3105 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:01:58.275091 3105 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:01:58.281721 3105 catalog_manager.cc:1347] Generated new cluster ID: c2a8244619e34f46bd68b9feeb8be937
I20260807 00:01:58.281808 3105 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:01:58.292155 3105 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:01:58.293381 3105 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:01:58.304674 3105 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1dd4c9d8142846deb019464b672ac565: Generated new TSK 0
I20260807 00:01:58.305540 3105 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:01:58.343269 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39247
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:58.465413 3119 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:58.465754 3119 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:58.465803 3119 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:58.470036 3119 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:58.470135 3119 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:58.470222 3119 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:01:58.476153 3119 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3119
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:58.477329 3119 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:58.478397 3119 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:58.488799 3125 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:58.488857 3130 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:58.489111 3127 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:58.489248 3119 server_base.cc:1034] running on GCE node
I20260807 00:01:58.489730 3119 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:58.490435 3119 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:58.491612 3119 hybrid_clock.cc:648] HybridClock initialized: now 1786060918491567 us; error 57 us; skew 500 ppm
Aug 07 00:01:58 dist-test-slave-pww0 krb5kdc[2961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:58.494437 3119 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:01:58.495448 3119 webserver.cc:460] Webserver started at http://127.30.204.65:33195/ using document root <none> and password file <none>
I20260807 00:01:58.496101 3119 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:58.496170 3119 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:58.496373 3119 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:58.498716 3119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "77e2219d4dab434b889b9375cef11b04"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "c22388855c8972b8496d398100d3a985"
server_key_iv: "af8c9fb289479be48988fb0fc83bab71"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.499347 3119 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "77e2219d4dab434b889b9375cef11b04"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "c22388855c8972b8496d398100d3a985"
server_key_iv: "af8c9fb289479be48988fb0fc83bab71"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.503504 3119 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:01:58.511003 3143 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.512895 3119 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.002s sys 0.000s
I20260807 00:01:58.513072 3119 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "77e2219d4dab434b889b9375cef11b04"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "c22388855c8972b8496d398100d3a985"
server_key_iv: "af8c9fb289479be48988fb0fc83bab71"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.513224 3119 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:58.533226 3119 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:58.536652 3119 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:58.536867 3119 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:58.537560 3119 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:58.538488 3119 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:58.538540 3119 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.538597 3119 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:58.538626 3119 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.547660 3119 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:46131
I20260807 00:01:58.547736 3312 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:46131 every 8 connection(s)
I20260807 00:01:58.548712 3119 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:01:58.551731 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 3119
I20260807 00:01:58.551857 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:01:58.552170 31537 external_mini_cluster.cc:1428] Setting key e809a2af76a35892634713ab2af983af
Aug 07 00:01:58 dist-test-slave-pww0 krb5kdc[2961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:01:58.564204 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:58.550769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:45685 (local address 127.30.204.126:39247)
0807 00:01:58.550926 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:58.550931 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:58.551878 (+ 947us) server_negotiation.cc:407] Connection header received
0807 00:01:58.553521 (+ 1643us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:58.553525 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:58.553587 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:58.553692 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:58.555034 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.555629 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.556848 (+ 1219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.557091 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.557194 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:58.560233 (+ 3039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:58.560238 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:58.560241 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:58.560279 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:58.561920 (+ 1641us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:58.562496 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:58.562500 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:58.562502 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:58.562600 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:58.562929 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:58.562933 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:58.562935 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:58.563210 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:58.563319 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:58.563899 (+ 580us) server_negotiation.cc:299] Negotiation successful
0807 00:01:58.564055 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:01:58.565109 3316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:58.551150 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39247 (local address 127.30.204.65:45685)
0807 00:01:58.551719 (+ 569us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:58.551767 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:58.553242 (+ 1475us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:58.553912 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:58.553921 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:58.554322 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:58.554873 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:58.554888 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.555771 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.555774 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:58.556717 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:58.556729 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.557264 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.557269 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:58.557494 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:58.558174 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:58.558197 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:58.560004 (+ 1807us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:58.562064 (+ 2060us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:58.562070 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:58.562087 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:58.562389 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:58.562692 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:58.562694 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:58.562695 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:58.562842 (+ 147us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:58.563310 (+ 468us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:58.563315 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:58.563627 (+ 312us) client_negotiation.cc:769] Sending connection context
0807 00:01:58.563832 (+ 205us) client_negotiation.cc:240] Negotiation successful
0807 00:01:58.564159 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":125,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:01:58.566644 3314 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39247
I20260807 00:01:58.566980 3314 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:58.567725 3314 heartbeater.cc:511] Master 127.30.204.126:39247 requested a full tablet report, sending...
I20260807 00:01:58.569785 3032 ts_manager.cc:194] Registered new tserver with Master: 77e2219d4dab434b889b9375cef11b04 (127.30.204.65:46131)
I20260807 00:01:58.571240 3032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:45685
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:01:58.584736 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39247
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:58.703689 3319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:58.704018 3319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:58.704093 3319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:58.708402 3319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:58.708490 3319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:58.708575 3319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:01:58.714484 3319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:58.715821 3319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:58.717027 3319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:58.727905 3330 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:58.727957 3324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:58.727905 3326 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:58.727928 3319 server_base.cc:1034] running on GCE node
I20260807 00:01:58.728686 3319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:58.729430 3319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:58.730608 3319 hybrid_clock.cc:648] HybridClock initialized: now 1786060918730580 us; error 41 us; skew 500 ppm
Aug 07 00:01:58 dist-test-slave-pww0 krb5kdc[2961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:58.733781 3319 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:01:58.734908 3319 webserver.cc:460] Webserver started at http://127.30.204.66:35239/ using document root <none> and password file <none>
I20260807 00:01:58.735596 3319 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:58.735689 3319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:58.735925 3319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:58.738236 3319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "e6f100c164cb4355a6057461ce8f1794"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "3160c8dc067f1d4b2e2582e3b60bc863"
server_key_iv: "b9455739a6de00d43ab66cabd1bf34c1"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.738893 3319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "e6f100c164cb4355a6057461ce8f1794"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "3160c8dc067f1d4b2e2582e3b60bc863"
server_key_iv: "b9455739a6de00d43ab66cabd1bf34c1"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.743091 3319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:01:58.745870 3340 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.747272 3319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:01:58.747417 3319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "e6f100c164cb4355a6057461ce8f1794"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "3160c8dc067f1d4b2e2582e3b60bc863"
server_key_iv: "b9455739a6de00d43ab66cabd1bf34c1"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.747591 3319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:58.764232 3319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:58.768049 3319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:58.768271 3319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:58.768951 3319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:58.769930 3319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:58.769984 3319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.770044 3319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:58.770085 3319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.779657 3319 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:39467
I20260807 00:01:58.779700 3511 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:39467 every 8 connection(s)
I20260807 00:01:58.780695 3319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:01:58.782948 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 3319
I20260807 00:01:58.783059 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:01:58.783362 31537 external_mini_cluster.cc:1428] Setting key 1b4ae2f62c553761040fa8c99c21e249
Aug 07 00:01:58 dist-test-slave-pww0 krb5kdc[2961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:01:58.797142 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:58.782664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:59389 (local address 127.30.204.126:39247)
0807 00:01:58.782790 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:58.782794 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:58.784076 (+ 1282us) server_negotiation.cc:407] Connection header received
0807 00:01:58.785676 (+ 1600us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:58.785679 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:58.785735 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:58.785806 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:58.787184 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.787805 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.789502 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.789752 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.789846 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:58.793055 (+ 3209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:58.793058 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:58.793060 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:58.793093 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:58.794664 (+ 1571us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:58.795271 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:58.795274 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:58.795276 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:58.795373 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:58.795774 (+ 401us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:58.795778 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:58.795779 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:58.795986 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:58.796081 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:58.796744 (+ 663us) server_negotiation.cc:299] Negotiation successful
0807 00:01:58.796899 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:01:58.798249 3514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:58.783259 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39247 (local address 127.30.204.66:59389)
0807 00:01:58.783910 (+ 651us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:58.783983 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:58.785450 (+ 1467us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:58.786001 (+ 551us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:58.786009 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:58.786400 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:58.786973 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:58.786997 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.787941 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.787945 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:58.789325 (+ 1380us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:58.789338 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:58.789951 (+ 613us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:58.789957 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:58.790194 (+ 237us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:58.790904 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:58.790927 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:58.792813 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:58.794809 (+ 1996us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:58.794817 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:58.794836 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:58.795156 (+ 320us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:58.795488 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:58.795491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:58.795493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:58.795670 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:58.796095 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:58.796104 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:58.796439 (+ 335us) client_negotiation.cc:769] Sending connection context
0807 00:01:58.796661 (+ 222us) client_negotiation.cc:240] Negotiation successful
0807 00:01:58.797087 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":216,"threads_started":1}
I20260807 00:01:58.799811 3512 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39247
I20260807 00:01:58.800082 3512 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:58.800642 3512 heartbeater.cc:511] Master 127.30.204.126:39247 requested a full tablet report, sending...
I20260807 00:01:58.801887 3032 ts_manager.cc:194] Registered new tserver with Master: e6f100c164cb4355a6057461ce8f1794 (127.30.204.66:39467)
I20260807 00:01:58.802599 3032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:59389
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:01:58.814733 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39247
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41017
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:01:58.938186 3520 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:01:58.938524 3520 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:01:58.938591 3520 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:01:58.942901 3520 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:01:58.942986 3520 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:01:58.943073 3520 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:01:58.948925 3520 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41017
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39247
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3520
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:01:58.950228 3520 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:01:58.951433 3520 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:01:58.962203 3533 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:01:58.962186 3526 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:58.962359 3520 server_base.cc:1034] running on GCE node
W20260807 00:01:58.962184 3528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:01:58.962996 3520 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:01:58.963792 3520 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:01:58.964996 3520 hybrid_clock.cc:648] HybridClock initialized: now 1786060918964972 us; error 42 us; skew 500 ppm
Aug 07 00:01:58 dist-test-slave-pww0 krb5kdc[2961](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:01:58.968230 3520 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:01:58.969347 3520 webserver.cc:460] Webserver started at http://127.30.204.67:40359/ using document root <none> and password file <none>
I20260807 00:01:58.969990 3520 fs_manager.cc:359] Metadata directory not provided
I20260807 00:01:58.970070 3520 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:01:58.970315 3520 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:01:58.972831 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "7daff04bef294500863875005ed63790"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "cb60783111602b650671627e536eff1a"
server_key_iv: "8afd76479a24ccb918c19cd1c5154d20"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.973479 3520 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "7daff04bef294500863875005ed63790"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "cb60783111602b650671627e536eff1a"
server_key_iv: "8afd76479a24ccb918c19cd1c5154d20"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.977836 3520 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:01:58.980839 3544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:58.982308 3520 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:01:58.982466 3520 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "7daff04bef294500863875005ed63790"
format_stamp: "Formatted at 2026-08-07 00:01:58 on dist-test-slave-pww0"
server_key: "cb60783111602b650671627e536eff1a"
server_key_iv: "8afd76479a24ccb918c19cd1c5154d20"
server_key_version: "encryptionkey@0"
I20260807 00:01:58.982589 3520 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:01:59.023226 3520 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:01:59.026791 3520 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:01:59.026970 3520 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:01:59.027627 3520 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:01:59.028522 3520 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:01:59.028570 3520 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:59.028611 3520 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:01:59.028654 3520 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:01:59.037786 3520 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:35123
I20260807 00:01:59.037825 3715 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:35123 every 8 connection(s)
I20260807 00:01:59.038789 3520 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:01:59.043702 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 3520
I20260807 00:01:59.043819 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:01:59.044121 31537 external_mini_cluster.cc:1428] Setting key e14a521b3b4a014f2c5b48547944d530
Aug 07 00:01:59 dist-test-slave-pww0 krb5kdc[2961](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060918, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:01:59.053709 3087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:59.040909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:44829 (local address 127.30.204.126:39247)
0807 00:01:59.041061 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:59.041065 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:59.042011 (+ 946us) server_negotiation.cc:407] Connection header received
0807 00:01:59.043433 (+ 1422us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:59.043437 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:59.043525 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:59.043602 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:59.045063 (+ 1461us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:59.045708 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:59.047006 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:01:59.047243 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:01:59.047329 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:59.050062 (+ 2733us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:01:59.050065 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:01:59.050067 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:01:59.050098 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:01:59.051543 (+ 1445us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:59.052081 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:59.052084 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:59.052085 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:59.052181 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:01:59.052590 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:01:59.052593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:01:59.052594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:01:59.052810 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:01:59.052903 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:01:59.053435 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 00:01:59.053573 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:01:59.054620 3718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:01:59.041294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39247 (local address 127.30.204.67:44829)
0807 00:01:59.041828 (+ 534us) negotiation.cc:107] Waiting for socket to connect
0807 00:01:59.041877 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:01:59.043189 (+ 1312us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:01:59.043816 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:01:59.043827 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:01:59.044267 (+ 440us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:01:59.044887 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:59.044905 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:59.045841 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:59.045844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:59.046889 (+ 1045us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:01:59.046900 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:01:59.047400 (+ 500us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:01:59.047403 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:01:59.047629 (+ 226us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:01:59.048205 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:01:59.048222 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:01:59.049863 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:01:59.051675 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:59.051681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:59.051695 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:59.051968 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:59.052271 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:01:59.052273 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:01:59.052275 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:01:59.052474 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:01:59.052899 (+ 425us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:01:59.052904 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:01:59.053181 (+ 277us) client_negotiation.cc:769] Sending connection context
0807 00:01:59.053350 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 00:01:59.053670 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":133,"threads_started":1}
I20260807 00:01:59.056031 3716 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39247
I20260807 00:01:59.056304 3716 heartbeater.cc:464] Registering TS with master...
I20260807 00:01:59.056883 3716 heartbeater.cc:511] Master 127.30.204.126:39247 requested a full tablet report, sending...
I20260807 00:01:59.058068 3032 ts_manager.cc:194] Registered new tserver with Master: 7daff04bef294500863875005ed63790 (127.30.204.67:35123)
I20260807 00:01:59.058712 3032 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:44829
I20260807 00:01:59.070724 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 00:01:59.086027 3087 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:01:59.084455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33028 (local address 127.30.204.126:39247)
0807 00:01:59.084584 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 00:01:59.084588 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:01:59.084713 (+ 125us) server_negotiation.cc:407] Connection header received
0807 00:01:59.084838 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:01:59.084841 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:01:59.084887 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:01:59.084971 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:01:59.085891 (+ 920us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33028: BlockingRecv error: recv got EOF from 127.0.0.1:33028 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:01:59.087103 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 3119
I20260807 00:01:59.115875 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 3319
I20260807 00:01:59.133395 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 3520
I20260807 00:01:59.155649 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2979
2026-08-07T00:01:59Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (2413 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:01:59 dist-test-slave-pww0 krb5kdc[3736](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:01:59 dist-test-slave-pww0 krb5kdc[3736](info): set up 2 sockets
Aug 07 00:01:59 dist-test-slave-pww0 krb5kdc[3736](info): commencing operation
krb5kdc: starting...
W20260807 00:02:00.339820 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:00 dist-test-slave-pww0 krb5kdc[3736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:00.478114 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39803
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42251
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:39803
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:00.600062 3757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:00.600411 3757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:00.600471 3757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:00.604755 3757 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:00.604846 3757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:00.604873 3757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:00.604894 3757 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:00.604916 3757 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:00.610877 3757 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42251
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:39803
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39803
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:00.612198 3757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:00.613320 3757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:00.622805 3767 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:00.622797 3766 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:00.622798 3770 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:00.623044 3757 server_base.cc:1034] running on GCE node
I20260807 00:02:00.623802 3757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:00.625059 3757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:00.626267 3757 hybrid_clock.cc:648] HybridClock initialized: now 1786060920626246 us; error 32 us; skew 500 ppm
Aug 07 00:02:00 dist-test-slave-pww0 krb5kdc[3736](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060920, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:00.633106 3757 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:00.634249 3757 webserver.cc:460] Webserver started at http://127.30.204.126:43743/ using document root <none> and password file <none>
I20260807 00:02:00.634881 3757 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:00.634961 3757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:00.635208 3757 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:00.637598 3757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "c25faaa01da04c10a25b7899b193968f"
format_stamp: "Formatted at 2026-08-07 00:02:00 on dist-test-slave-pww0"
server_key: "1dbf41f64534a30a0a87b29799c0621e"
server_key_iv: "1dabfd4c3cdd650120613461e8a9bf55"
server_key_version: "encryptionkey@0"
I20260807 00:02:00.638257 3757 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "c25faaa01da04c10a25b7899b193968f"
format_stamp: "Formatted at 2026-08-07 00:02:00 on dist-test-slave-pww0"
server_key: "1dbf41f64534a30a0a87b29799c0621e"
server_key_iv: "1dabfd4c3cdd650120613461e8a9bf55"
server_key_version: "encryptionkey@0"
I20260807 00:02:00.642668 3757 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:00.645758 3779 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:00.647323 3757 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:00.647518 3757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "c25faaa01da04c10a25b7899b193968f"
format_stamp: "Formatted at 2026-08-07 00:02:00 on dist-test-slave-pww0"
server_key: "1dbf41f64534a30a0a87b29799c0621e"
server_key_iv: "1dabfd4c3cdd650120613461e8a9bf55"
server_key_version: "encryptionkey@0"
I20260807 00:02:00.647666 3757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:00.666903 3757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:00.670528 3757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:00.670753 3757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:00.678421 3757 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:39803
I20260807 00:02:00.678740 3851 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:39803 every 8 connection(s)
I20260807 00:02:00.679530 3757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:00.682586 3852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:00.686616 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 3757
I20260807 00:02:00.686743 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:00.687007 31537 external_mini_cluster.cc:1428] Setting key 37956bdc6f1e892020ad98bdb3ea4834
I20260807 00:02:00.689316 3852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f: Bootstrap starting.
I20260807 00:02:00.692442 3852 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:00.693457 3852 log.cc:824] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:00.696110 3852 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f: No bootstrap required, opened a new log
Aug 07 00:02:00 dist-test-slave-pww0 krb5kdc[3736](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060920, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:00.700589 3852 raft_consensus.cc:348] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25faaa01da04c10a25b7899b193968f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39803 } }
I20260807 00:02:00.700901 3852 raft_consensus.cc:374] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:00.700971 3852 raft_consensus.cc:729] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c25faaa01da04c10a25b7899b193968f, State: Initialized, Role: FOLLOWER
I20260807 00:02:00.701498 3852 consensus_queue.cc:260] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25faaa01da04c10a25b7899b193968f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39803 } }
I20260807 00:02:00.701644 3852 raft_consensus.cc:388] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:00.701712 3852 raft_consensus.cc:482] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:00.701818 3852 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:00.702898 3852 raft_consensus.cc:504] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25faaa01da04c10a25b7899b193968f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39803 } }
I20260807 00:02:00.703279 3852 leader_election.cc:302] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c25faaa01da04c10a25b7899b193968f; no voters:
I20260807 00:02:00.703653 3852 leader_election.cc:288] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:00.703773 3859 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:00.704025 3859 raft_consensus.cc:686] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [term 1 LEADER]: Becoming Leader. State: Replica: c25faaa01da04c10a25b7899b193968f, State: Running, Role: LEADER
I20260807 00:02:00.704095 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:00.688514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41154 (local address 127.30.204.126:39803)
0807 00:02:00.689039 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:00.689058 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:00.689104 (+ 46us) server_negotiation.cc:407] Connection header received
0807 00:02:00.689982 (+ 878us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:00.690008 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:00.690486 (+ 478us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:00.690926 (+ 440us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:00.691751 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:00.693254 (+ 1503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:00.694372 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:00.695079 (+ 707us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:00.695261 (+ 182us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:00.697990 (+ 2729us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:00.697997 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:00.698019 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:00.698058 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:00.700209 (+ 2151us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:00.700813 (+ 604us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:00.700819 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:00.700826 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:00.700993 (+ 167us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:00.701351 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:00.701355 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:00.701357 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:00.701949 (+ 592us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:00.702168 (+ 219us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:00.702502 (+ 334us) server_negotiation.cc:299] Negotiation successful
0807 00:02:00.702846 (+ 344us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":353,"thread_start_us":159,"threads_started":1}
I20260807 00:02:00.704487 3859 consensus_queue.cc:237] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25faaa01da04c10a25b7899b193968f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39803 } }
I20260807 00:02:00.705049 3852 sys_catalog.cc:564] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:00.706277 3862 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [sys.catalog]: SysCatalogTable state changed. Reason: New leader c25faaa01da04c10a25b7899b193968f. Latest consensus state: current_term: 1 leader_uuid: "c25faaa01da04c10a25b7899b193968f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25faaa01da04c10a25b7899b193968f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39803 } } }
I20260807 00:02:00.706404 3862 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:00.706704 3860 sys_catalog.cc:455] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c25faaa01da04c10a25b7899b193968f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c25faaa01da04c10a25b7899b193968f" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39803 } } }
I20260807 00:02:00.706810 3860 sys_catalog.cc:458] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:00.706902 3870 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:00.707968 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:02:00.710682 3870 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:00.717260 3870 catalog_manager.cc:1347] Generated new cluster ID: 902099ba6d5e41579c5cfe941d459f3c
I20260807 00:02:00.717412 3870 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:00.733549 3870 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:00.734684 3857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:00.726840 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.30.204.126:39803)
0807 00:02:00.726980 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:00.726984 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:00.727224 (+ 240us) server_negotiation.cc:407] Connection header received
0807 00:02:00.727361 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:00.727364 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:00.727413 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:00.727516 (+ 103us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:00.728087 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:00.728658 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:00.729505 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:00.729757 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:00.729850 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:00.733485 (+ 3635us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:00.733489 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:00.733492 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:00.734116 (+ 624us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:00.734161 (+ 45us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:00.734182 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:00.734274 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:00.734414 (+ 140us) server_negotiation.cc:299] Negotiation successful
0807 00:02:00.734499 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:02:00.734980 3870 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:00.744750 3870 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c25faaa01da04c10a25b7899b193968f: Generated new TSK 0
I20260807 00:02:00.745615 3870 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:02:00.753456 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 3757
2026-08-07T00:02:00Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1603 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:00 dist-test-slave-pww0 krb5kdc[3901](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:00 dist-test-slave-pww0 krb5kdc[3901](info): set up 2 sockets
Aug 07 00:02:00 dist-test-slave-pww0 krb5kdc[3901](info): commencing operation
krb5kdc: starting...
W20260807 00:02:01.929948 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:02 dist-test-slave-pww0 krb5kdc[3901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:02.065933 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43719
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:43719
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:02.190249 3921 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:02.190629 3921 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:02.190697 3921 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:02.195013 3921 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:02.195107 3921 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:02.195137 3921 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:02.195160 3921 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:02.195180 3921 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:02.201117 3921 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:43719
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43719
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3921
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:02.202400 3921 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:02.203749 3921 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:02.213377 3933 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:02.213383 3935 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:02.213377 3937 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:02.213474 3921 server_base.cc:1034] running on GCE node
I20260807 00:02:02.214350 3921 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:02.215694 3921 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:02.216893 3921 hybrid_clock.cc:648] HybridClock initialized: now 1786060922216871 us; error 38 us; skew 500 ppm
Aug 07 00:02:02 dist-test-slave-pww0 krb5kdc[3901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:02.220088 3921 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:02.221155 3921 webserver.cc:460] Webserver started at http://127.30.204.126:42357/ using document root <none> and password file <none>
I20260807 00:02:02.221763 3921 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:02.221845 3921 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:02.222075 3921 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:02.224486 3921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "0935275192c74962b21c6550f4178bca"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "dd3e007ef473323cd4cca7f80630ed84"
server_key_iv: "d8f62ee58ee3f7a259a119902b5d2bed"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.225131 3921 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "0935275192c74962b21c6550f4178bca"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "dd3e007ef473323cd4cca7f80630ed84"
server_key_iv: "d8f62ee58ee3f7a259a119902b5d2bed"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.229554 3921 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:02.232388 3950 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:02.233798 3921 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:02.233996 3921 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "0935275192c74962b21c6550f4178bca"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "dd3e007ef473323cd4cca7f80630ed84"
server_key_iv: "d8f62ee58ee3f7a259a119902b5d2bed"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.234122 3921 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:02.248968 3921 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:02.252436 3921 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:02.252627 3921 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:02.260578 3921 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:43719
I20260807 00:02:02.260881 4021 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:43719 every 8 connection(s)
I20260807 00:02:02.261682 3921 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:02.264386 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 3921
I20260807 00:02:02.264510 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:02.264794 31537 external_mini_cluster.cc:1428] Setting key f7142a54de591816fee68dd22c1ac7ae
I20260807 00:02:02.264961 4026 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:02.272408 4026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca: Bootstrap starting.
Aug 07 00:02:02 dist-test-slave-pww0 krb5kdc[3901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060922, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:02.275413 4026 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:02.276398 4026 log.cc:824] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:02.278914 4026 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca: No bootstrap required, opened a new log
I20260807 00:02:02.280860 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:02.266303 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49806 (local address 127.30.204.126:43719)
0807 00:02:02.266733 (+ 430us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:02.266757 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:02.266794 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:02:02.267873 (+ 1079us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:02.267895 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:02.268256 (+ 361us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:02.268554 (+ 298us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:02.269546 (+ 992us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.270626 (+ 1080us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.271837 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.272341 (+ 504us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.272466 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:02.275409 (+ 2943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:02.275415 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:02.275431 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:02.275509 (+ 78us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:02.277322 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:02.277873 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:02.277878 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:02.277884 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:02.278019 (+ 135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:02.278366 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:02.278369 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:02.278371 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:02.278844 (+ 473us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:02.279010 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:02.279417 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 00:02:02.279830 (+ 413us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":282,"thread_start_us":117,"threads_started":1}
I20260807 00:02:02.282322 4026 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0935275192c74962b21c6550f4178bca" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43719 } }
I20260807 00:02:02.282577 4026 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:02.282613 4026 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0935275192c74962b21c6550f4178bca, State: Initialized, Role: FOLLOWER
I20260807 00:02:02.283054 4026 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0935275192c74962b21c6550f4178bca" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43719 } }
I20260807 00:02:02.283214 4026 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:02.283336 4026 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:02.283452 4026 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:02.284513 4026 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0935275192c74962b21c6550f4178bca" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43719 } }
I20260807 00:02:02.284859 4026 leader_election.cc:302] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0935275192c74962b21c6550f4178bca; no voters:
I20260807 00:02:02.285178 4026 leader_election.cc:288] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:02.285322 4037 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:02.285574 4037 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [term 1 LEADER]: Becoming Leader. State: Replica: 0935275192c74962b21c6550f4178bca, State: Running, Role: LEADER
I20260807 00:02:02.285948 4037 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0935275192c74962b21c6550f4178bca" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43719 } }
I20260807 00:02:02.286669 4026 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:02.287866 4038 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0935275192c74962b21c6550f4178bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0935275192c74962b21c6550f4178bca" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43719 } } }
I20260807 00:02:02.288012 4038 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:02.288326 4041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0935275192c74962b21c6550f4178bca. Latest consensus state: current_term: 1 leader_uuid: "0935275192c74962b21c6550f4178bca" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0935275192c74962b21c6550f4178bca" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43719 } } }
I20260807 00:02:02.288419 4056 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:02.288516 4041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:02.291922 4056 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:02.298326 4056 catalog_manager.cc:1347] Generated new cluster ID: 462be3e3c0c0426789d4b14feae31fb9
I20260807 00:02:02.298415 4056 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:02.310770 4056 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:02.311911 4056 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:02.320453 4056 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0935275192c74962b21c6550f4178bca: Generated new TSK 0
I20260807 00:02:02.321269 4056 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:02:02.364956 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43719
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:02.498828 4074 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:02.499125 4074 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:02.499169 4074 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:02.503284 4074 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:02.503367 4074 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:02.503429 4074 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:02.509073 4074 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4074
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:02.510238 4074 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:02.511319 4074 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:02.522348 4083 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:02.522361 4082 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:02.522348 4087 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:02.522848 4074 server_base.cc:1034] running on GCE node
I20260807 00:02:02.523381 4074 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:02.524123 4074 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:02.525296 4074 hybrid_clock.cc:648] HybridClock initialized: now 1786060922525274 us; error 38 us; skew 500 ppm
Aug 07 00:02:02 dist-test-slave-pww0 krb5kdc[3901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:02.528301 4074 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:02:02.529430 4074 webserver.cc:460] Webserver started at http://127.30.204.65:45009/ using document root <none> and password file <none>
I20260807 00:02:02.530018 4074 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:02.530090 4074 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:02.530325 4074 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:02.532722 4074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "14c2eceef5ed43b1b0bc2d1f5b0e839e"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "7bf9d9308e40c206f5624aba57279b5f"
server_key_iv: "bcf5205a90b5dc8a1d36bfa9893aa2c3"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.533344 4074 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "14c2eceef5ed43b1b0bc2d1f5b0e839e"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "7bf9d9308e40c206f5624aba57279b5f"
server_key_iv: "bcf5205a90b5dc8a1d36bfa9893aa2c3"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.537766 4074 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:02.540473 4098 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:02.541863 4074 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:02.541996 4074 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "14c2eceef5ed43b1b0bc2d1f5b0e839e"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "7bf9d9308e40c206f5624aba57279b5f"
server_key_iv: "bcf5205a90b5dc8a1d36bfa9893aa2c3"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.542109 4074 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:02.573521 4074 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:02.577255 4074 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:02.577491 4074 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:02.578187 4074 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:02.579159 4074 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:02.579212 4074 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:02.579272 4074 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:02.579303 4074 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:02.589140 4074 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:33633
I20260807 00:02:02.589193 4265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:33633 every 8 connection(s)
I20260807 00:02:02.590162 4074 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:02.594640 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 4074
I20260807 00:02:02.594794 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:02.595065 31537 external_mini_cluster.cc:1428] Setting key 51d3f31aa46ae82cdf4860907d0db175
Aug 07 00:02:02 dist-test-slave-pww0 krb5kdc[3901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:02.605221 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:02.592256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:52961 (local address 127.30.204.126:43719)
0807 00:02:02.592400 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:02.592405 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:02.593352 (+ 947us) server_negotiation.cc:407] Connection header received
0807 00:02:02.594848 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:02.594852 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:02.594918 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:02.594995 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:02.596552 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.597101 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.598328 (+ 1227us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.598581 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.598685 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:02.601382 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:02.601386 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:02.601388 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:02.601418 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:02.602895 (+ 1477us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:02.603494 (+ 599us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:02.603498 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:02.603500 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:02.603592 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:02.603981 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:02.603985 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:02.603987 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:02.604203 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:02.604296 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:02.604883 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 00:02:02.605041 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 00:02:02.606172 4271 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:02.592665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43719 (local address 127.30.204.65:52961)
0807 00:02:02.593179 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:02.593229 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:02.594570 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:02.595228 (+ 658us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:02.595237 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:02.595765 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:02.596397 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:02.596420 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.597239 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.597242 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:02.598184 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:02.598196 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.598752 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.598757 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:02.598946 (+ 189us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:02.599591 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:02.599610 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:02.601180 (+ 1570us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:02.603027 (+ 1847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:02.603034 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:02.603055 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:02.603354 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:02.603709 (+ 355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:02.603712 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:02.603714 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:02.603883 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:02.604317 (+ 434us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:02.604326 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:02.604618 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 00:02:02.604816 (+ 198us) client_negotiation.cc:240] Negotiation successful
0807 00:02:02.605219 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":116,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:02:02.607609 4268 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43719
I20260807 00:02:02.607895 4268 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:02.608556 4268 heartbeater.cc:511] Master 127.30.204.126:43719 requested a full tablet report, sending...
I20260807 00:02:02.610553 3975 ts_manager.cc:194] Registered new tserver with Master: 14c2eceef5ed43b1b0bc2d1f5b0e839e (127.30.204.65:33633)
I20260807 00:02:02.611893 3975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:52961
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:02:02.626145 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43719
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:02.744634 4275 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:02.744935 4275 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:02.744982 4275 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:02.749222 4275 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:02.749300 4275 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:02.749361 4275 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:02.755103 4275 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4275
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:02.756332 4275 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:02.757422 4275 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:02:02.768249 4275 server_base.cc:1034] running on GCE node
W20260807 00:02:02.768134 4283 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:02.768134 4282 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:02.768136 4289 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:02.768927 4275 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:02.769614 4275 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:02.770802 4275 hybrid_clock.cc:648] HybridClock initialized: now 1786060922770780 us; error 36 us; skew 500 ppm
Aug 07 00:02:02 dist-test-slave-pww0 krb5kdc[3901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:02.773937 4275 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:02:02.774945 4275 webserver.cc:460] Webserver started at http://127.30.204.66:46295/ using document root <none> and password file <none>
I20260807 00:02:02.775581 4275 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:02.775661 4275 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:02.775893 4275 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:02.778105 4275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "fe323ca23eb84770a463e6087e4ca091"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "4a3c3c0dcb13b3f6496ee0c275166049"
server_key_iv: "0ad750dc767bf9de00ed114737eaf688"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.778717 4275 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "fe323ca23eb84770a463e6087e4ca091"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "4a3c3c0dcb13b3f6496ee0c275166049"
server_key_iv: "0ad750dc767bf9de00ed114737eaf688"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.782824 4275 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:02.785619 4297 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:02.787104 4275 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:02.787290 4275 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "fe323ca23eb84770a463e6087e4ca091"
format_stamp: "Formatted at 2026-08-07 00:02:02 on dist-test-slave-pww0"
server_key: "4a3c3c0dcb13b3f6496ee0c275166049"
server_key_iv: "0ad750dc767bf9de00ed114737eaf688"
server_key_version: "encryptionkey@0"
I20260807 00:02:02.787490 4275 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:02.818269 4275 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:02.822202 4275 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:02.822412 4275 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:02.823108 4275 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:02.824091 4275 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:02.824162 4275 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:02.824218 4275 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:02.824240 4275 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:02.833398 4275 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:42663
I20260807 00:02:02.833462 4461 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:42663 every 8 connection(s)
I20260807 00:02:02.834560 4275 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:02.844725 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 4275
I20260807 00:02:02.844849 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:02.845132 31537 external_mini_cluster.cc:1428] Setting key 60161627e13999dc6344cae85f3c4a63
Aug 07 00:02:02 dist-test-slave-pww0 krb5kdc[3901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060922, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:02.850096 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:02.836633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:36949 (local address 127.30.204.126:43719)
0807 00:02:02.836774 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:02.836778 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:02.837728 (+ 950us) server_negotiation.cc:407] Connection header received
0807 00:02:02.839244 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:02.839248 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:02.839313 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:02.839390 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:02.840719 (+ 1329us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.841267 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.842701 (+ 1434us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.842950 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.843049 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:02.846029 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:02.846033 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:02.846036 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:02.846061 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:02.847679 (+ 1618us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:02.848233 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:02.848236 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:02.848237 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:02.848336 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:02.848735 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:02.848738 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:02.848739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:02.848986 (+ 247us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:02.849091 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:02.849837 (+ 746us) server_negotiation.cc:299] Negotiation successful
0807 00:02:02.849965 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:02:02.851219 4466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:02.836981 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43719 (local address 127.30.204.66:36949)
0807 00:02:02.837557 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:02.837616 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:02.839002 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:02.839615 (+ 613us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:02.839623 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:02.840018 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:02.840557 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:02.840572 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.841567 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.841570 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:02.842557 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:02.842569 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:02.843125 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:02.843130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:02.843318 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:02.843983 (+ 665us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:02.844002 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:02.845828 (+ 1826us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:02.847816 (+ 1988us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:02.847823 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:02.847839 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:02.848134 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:02.848453 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:02.848456 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:02.848458 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:02.848640 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:02.849130 (+ 490us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:02.849138 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:02.849535 (+ 397us) client_negotiation.cc:769] Sending connection context
0807 00:02:02.849775 (+ 240us) client_negotiation.cc:240] Negotiation successful
0807 00:02:02.850194 (+ 419us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":126,"threads_started":1}
I20260807 00:02:02.852826 4464 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43719
I20260807 00:02:02.853121 4464 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:02.853724 4464 heartbeater.cc:511] Master 127.30.204.126:43719 requested a full tablet report, sending...
I20260807 00:02:02.855001 3975 ts_manager.cc:194] Registered new tserver with Master: fe323ca23eb84770a463e6087e4ca091 (127.30.204.66:42663)
I20260807 00:02:02.855645 3975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:36949
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:02:02.878386 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43719
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46219
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:02.998790 4469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:02.999094 4469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:02.999140 4469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:03.003405 4469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:03.003535 4469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:03.003604 4469 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:03.009316 4469 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46219
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43719
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:03.010623 4469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:03.011744 4469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:03.022078 4476 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:03.022163 4469 server_base.cc:1034] running on GCE node
W20260807 00:02:03.022078 4477 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:03.022079 4481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:03.022764 4469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:03.023530 4469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:03.024724 4469 hybrid_clock.cc:648] HybridClock initialized: now 1786060923024695 us; error 52 us; skew 500 ppm
Aug 07 00:02:03 dist-test-slave-pww0 krb5kdc[3901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060923, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:03.027757 4469 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:02:03.028776 4469 webserver.cc:460] Webserver started at http://127.30.204.67:44301/ using document root <none> and password file <none>
I20260807 00:02:03.029361 4469 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:03.029433 4469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:03.029655 4469 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:03.031939 4469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "5d0ace36fa6644248642a686b8c76d8d"
format_stamp: "Formatted at 2026-08-07 00:02:03 on dist-test-slave-pww0"
server_key: "05f28ff869e5d7500eae3f0f0c7dd809"
server_key_iv: "5c134685a27623d87ab703edb727abaf"
server_key_version: "encryptionkey@0"
I20260807 00:02:03.032550 4469 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "5d0ace36fa6644248642a686b8c76d8d"
format_stamp: "Formatted at 2026-08-07 00:02:03 on dist-test-slave-pww0"
server_key: "05f28ff869e5d7500eae3f0f0c7dd809"
server_key_iv: "5c134685a27623d87ab703edb727abaf"
server_key_version: "encryptionkey@0"
I20260807 00:02:03.036728 4469 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:03.039691 4488 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:03.041220 4469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:02:03.041383 4469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "5d0ace36fa6644248642a686b8c76d8d"
format_stamp: "Formatted at 2026-08-07 00:02:03 on dist-test-slave-pww0"
server_key: "05f28ff869e5d7500eae3f0f0c7dd809"
server_key_iv: "5c134685a27623d87ab703edb727abaf"
server_key_version: "encryptionkey@0"
I20260807 00:02:03.041505 4469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:03.058663 4469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:03.062114 4469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:03.062327 4469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:03.062976 4469 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:03.063927 4469 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:03.063980 4469 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:03.064039 4469 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:03.064065 4469 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:03.073069 4469 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:35221
I20260807 00:02:03.073112 4654 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:35221 every 8 connection(s)
I20260807 00:02:03.074057 4469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:03.076529 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 4469
I20260807 00:02:03.076673 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:03.077016 31537 external_mini_cluster.cc:1428] Setting key 2fd8a5d243cffd7a248415252657f223
Aug 07 00:02:03 dist-test-slave-pww0 krb5kdc[3901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060923, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:03.089368 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:03.076141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:49461 (local address 127.30.204.126:43719)
0807 00:02:03.076341 (+ 200us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:03.076346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:03.077478 (+ 1132us) server_negotiation.cc:407] Connection header received
0807 00:02:03.079080 (+ 1602us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:03.079083 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:03.079147 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:03.079236 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:03.080559 (+ 1323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:03.081093 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:03.082402 (+ 1309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:03.082636 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:03.082718 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:03.085604 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:03.085609 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:03.085612 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:03.085644 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:03.087076 (+ 1432us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:03.087691 (+ 615us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:03.087695 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:03.087696 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:03.087781 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:03.088221 (+ 440us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:03.088224 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:03.088225 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:03.088429 (+ 204us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:03.088505 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:03.089118 (+ 613us) server_negotiation.cc:299] Negotiation successful
0807 00:02:03.089236 (+ 118us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 00:02:03.090428 4657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:03.076677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43719 (local address 127.30.204.67:49461)
0807 00:02:03.077278 (+ 601us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:03.077333 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:03.078861 (+ 1528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:03.079441 (+ 580us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:03.079449 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:03.079892 (+ 443us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:03.080418 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:03.080433 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:03.081263 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:03.081266 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:03.082241 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:03.082262 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:03.082794 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:03.082798 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:03.082981 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:03.083746 (+ 765us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:03.083766 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:03.085381 (+ 1615us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:03.087205 (+ 1824us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:03.087211 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:03.087228 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:03.087600 (+ 372us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:03.087868 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:03.087873 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:03.087875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:03.088126 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:03.088571 (+ 445us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:03.088578 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:03.088859 (+ 281us) client_negotiation.cc:769] Sending connection context
0807 00:02:03.089033 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 00:02:03.089348 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":154,"threads_started":1}
I20260807 00:02:03.091733 4655 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43719
I20260807 00:02:03.091988 4655 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:03.092612 4655 heartbeater.cc:511] Master 127.30.204.126:43719 requested a full tablet report, sending...
I20260807 00:02:03.093765 3975 ts_manager.cc:194] Registered new tserver with Master: 5d0ace36fa6644248642a686b8c76d8d (127.30.204.67:35221)
I20260807 00:02:03.094347 3975 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:49461
I20260807 00:02:03.103593 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:02:03 dist-test-slave-pww0 krb5kdc[3901](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060923, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Aug 07 00:02:03 dist-test-slave-pww0 krb5kdc[3901](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060923, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:03.137863 4029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:03.122365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49818 (local address 127.30.204.126:43719)
0807 00:02:03.122563 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:03.122568 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:03.122688 (+ 120us) server_negotiation.cc:407] Connection header received
0807 00:02:03.122842 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:03.122845 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:03.122893 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:03.122965 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:03.123809 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:03.124350 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:03.125648 (+ 1298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:03.126011 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:03.126140 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:03.129242 (+ 3102us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:03.129246 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:03.129249 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:03.129287 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:03.135436 (+ 6149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:03.136152 (+ 716us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:03.136156 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:03.136158 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:03.136290 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:03.136645 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:03.136650 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:03.136651 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:03.136990 (+ 339us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:03.137149 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:03.137512 (+ 363us) server_negotiation.cc:299] Negotiation successful
0807 00:02:03.137677 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":82}
W20260807 00:02:03.138454 3975 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:49818
I20260807 00:02:03.139431 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 4074
I20260807 00:02:03.174759 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 4275
I20260807 00:02:03.191397 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 4469
I20260807 00:02:03.220706 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 3921
2026-08-07T00:02:03Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (2460 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:03 dist-test-slave-pww0 krb5kdc[4670](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:03 dist-test-slave-pww0 krb5kdc[4670](info): set up 2 sockets
Aug 07 00:02:03 dist-test-slave-pww0 krb5kdc[4670](info): commencing operation
krb5kdc: starting...
W20260807 00:02:04.374780 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:04 dist-test-slave-pww0 krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:04.507946 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37565
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:37565
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:04.626178 4693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:04.626507 4693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:04.626569 4693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:04.630846 4693 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:04.630944 4693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:04.630972 4693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:04.630993 4693 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:04.631016 4693 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:04.636767 4693 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:37565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:04.638021 4693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:04.639211 4693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:04.648484 4701 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:04.648591 4702 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:04.648656 4693 server_base.cc:1034] running on GCE node
W20260807 00:02:04.648527 4705 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:04.649492 4693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:04.650826 4693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:04.652014 4693 hybrid_clock.cc:648] HybridClock initialized: now 1786060924651981 us; error 50 us; skew 500 ppm
Aug 07 00:02:04 dist-test-slave-pww0 krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:04.655109 4693 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:04.656216 4693 webserver.cc:460] Webserver started at http://127.30.204.126:34117/ using document root <none> and password file <none>
I20260807 00:02:04.656788 4693 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:04.656852 4693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:04.657027 4693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:04.659597 4693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "5c1a6610753b4f8da6accc1d89023b12"
format_stamp: "Formatted at 2026-08-07 00:02:04 on dist-test-slave-pww0"
server_key: "a6e56757bb9aa8fd52f40ae1cd1a6218"
server_key_iv: "a132c5118b386172b4af73f2f31fd595"
server_key_version: "encryptionkey@0"
I20260807 00:02:04.660252 4693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "5c1a6610753b4f8da6accc1d89023b12"
format_stamp: "Formatted at 2026-08-07 00:02:04 on dist-test-slave-pww0"
server_key: "a6e56757bb9aa8fd52f40ae1cd1a6218"
server_key_iv: "a132c5118b386172b4af73f2f31fd595"
server_key_version: "encryptionkey@0"
I20260807 00:02:04.664562 4693 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:04.667384 4715 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:04.668815 4693 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:04.668948 4693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "5c1a6610753b4f8da6accc1d89023b12"
format_stamp: "Formatted at 2026-08-07 00:02:04 on dist-test-slave-pww0"
server_key: "a6e56757bb9aa8fd52f40ae1cd1a6218"
server_key_iv: "a132c5118b386172b4af73f2f31fd595"
server_key_version: "encryptionkey@0"
I20260807 00:02:04.669071 4693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:04.682068 4693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:04.685509 4693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:04.685721 4693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:04.693306 4693 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:37565
I20260807 00:02:04.693318 4796 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:37565 every 8 connection(s)
I20260807 00:02:04.694377 4693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:04.695745 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 4693
I20260807 00:02:04.695871 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:04.696151 31537 external_mini_cluster.cc:1428] Setting key 8ccf4d7d91b082d778de20cbe7304832
I20260807 00:02:04.697765 4798 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:04.704854 4798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12: Bootstrap starting.
Aug 07 00:02:04 dist-test-slave-pww0 krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060924, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:04.707578 4798 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:04.708384 4798 log.cc:824] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:04.710662 4798 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12: No bootstrap required, opened a new log
I20260807 00:02:04.711486 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:04.697607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54212 (local address 127.30.204.126:37565)
0807 00:02:04.698124 (+ 517us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:04.698156 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:04.698195 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:02:04.698854 (+ 659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:04.698871 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:04.699228 (+ 357us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:04.699576 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:04.700618 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:04.701667 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:04.702825 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:04.703319 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:04.703440 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:04.706247 (+ 2807us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:04.706253 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:04.706267 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:04.706301 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:04.708071 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:04.708786 (+ 715us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:04.708792 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:04.708798 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:04.708948 (+ 150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:04.709262 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:04.709265 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:04.709266 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:04.709732 (+ 466us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:04.709926 (+ 194us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:04.710258 (+ 332us) server_negotiation.cc:299] Negotiation successful
0807 00:02:04.710591 (+ 333us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":350,"thread_start_us":152,"threads_started":1}
I20260807 00:02:04.715055 4798 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1a6610753b4f8da6accc1d89023b12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37565 } }
I20260807 00:02:04.715353 4798 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:04.715420 4798 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c1a6610753b4f8da6accc1d89023b12, State: Initialized, Role: FOLLOWER
I20260807 00:02:04.715939 4798 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1a6610753b4f8da6accc1d89023b12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37565 } }
I20260807 00:02:04.716117 4798 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:04.716205 4798 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:04.716313 4798 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:04.717307 4798 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1a6610753b4f8da6accc1d89023b12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37565 } }
I20260807 00:02:04.717653 4798 leader_election.cc:302] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5c1a6610753b4f8da6accc1d89023b12; no voters:
I20260807 00:02:04.717954 4798 leader_election.cc:288] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:04.718086 4808 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:04.718334 4808 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [term 1 LEADER]: Becoming Leader. State: Replica: 5c1a6610753b4f8da6accc1d89023b12, State: Running, Role: LEADER
I20260807 00:02:04.718673 4808 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1a6610753b4f8da6accc1d89023b12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37565 } }
I20260807 00:02:04.719393 4798 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:04.720526 4811 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5c1a6610753b4f8da6accc1d89023b12. Latest consensus state: current_term: 1 leader_uuid: "5c1a6610753b4f8da6accc1d89023b12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1a6610753b4f8da6accc1d89023b12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37565 } } }
I20260807 00:02:04.720688 4811 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:04.720953 4809 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5c1a6610753b4f8da6accc1d89023b12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5c1a6610753b4f8da6accc1d89023b12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37565 } } }
I20260807 00:02:04.721052 4809 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:04.721172 4820 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:04.724681 4820 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:04.731228 4820 catalog_manager.cc:1347] Generated new cluster ID: e6aaddcf2a0e4b028a37baa6fcd5549f
I20260807 00:02:04.731325 4820 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:04.747013 4820 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:04.748349 4820 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:04.753471 4820 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5c1a6610753b4f8da6accc1d89023b12: Generated new TSK 0
I20260807 00:02:04.754434 4820 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:02:04.787212 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37565
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:04.912761 4841 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:04.913089 4841 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:04.913156 4841 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:04.917538 4841 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:04.917623 4841 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:04.917721 4841 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:04.923717 4841 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.4841
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:04.924979 4841 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:04.926131 4841 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:04.936951 4853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:04.937007 4849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:04.937007 4855 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:04.937542 4841 server_base.cc:1034] running on GCE node
I20260807 00:02:04.938010 4841 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:04.938750 4841 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:04.939944 4841 hybrid_clock.cc:648] HybridClock initialized: now 1786060924939918 us; error 38 us; skew 500 ppm
Aug 07 00:02:04 dist-test-slave-pww0 krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:04.943091 4841 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:02:04.944221 4841 webserver.cc:460] Webserver started at http://127.30.204.65:36241/ using document root <none> and password file <none>
I20260807 00:02:04.944850 4841 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:04.944926 4841 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:04.945165 4841 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:04.947599 4841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "29dbbc480257431db465fd7d1eea4cbb"
format_stamp: "Formatted at 2026-08-07 00:02:04 on dist-test-slave-pww0"
server_key: "19f228f867c7b4eb55a3ef0c7a03fd18"
server_key_iv: "0e453ed6e315c652556d8202903220cc"
server_key_version: "encryptionkey@0"
I20260807 00:02:04.948278 4841 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "29dbbc480257431db465fd7d1eea4cbb"
format_stamp: "Formatted at 2026-08-07 00:02:04 on dist-test-slave-pww0"
server_key: "19f228f867c7b4eb55a3ef0c7a03fd18"
server_key_iv: "0e453ed6e315c652556d8202903220cc"
server_key_version: "encryptionkey@0"
I20260807 00:02:04.952646 4841 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:02:04.955659 4869 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:04.957157 4841 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:04.957417 4841 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "29dbbc480257431db465fd7d1eea4cbb"
format_stamp: "Formatted at 2026-08-07 00:02:04 on dist-test-slave-pww0"
server_key: "19f228f867c7b4eb55a3ef0c7a03fd18"
server_key_iv: "0e453ed6e315c652556d8202903220cc"
server_key_version: "encryptionkey@0"
I20260807 00:02:04.957577 4841 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:04.982263 4841 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:04.985808 4841 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:04.986016 4841 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:04.986732 4841 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:04.987742 4841 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:04.987798 4841 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:04.987843 4841 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:04.987867 4841 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:04.997082 4841 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:45683
I20260807 00:02:04.997121 5048 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:45683 every 8 connection(s)
I20260807 00:02:04.998152 4841 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:05.006358 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 4841
I20260807 00:02:05.006480 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:05.006771 31537 external_mini_cluster.cc:1428] Setting key 33d802d24ded9ec17f89c5265029d732
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060924, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:05.013624 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.000219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:56039 (local address 127.30.204.126:37565)
0807 00:02:05.000358 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:05.000362 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:05.001336 (+ 974us) server_negotiation.cc:407] Connection header received
0807 00:02:05.002912 (+ 1576us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:05.002916 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:05.002983 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:05.003062 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:05.004542 (+ 1480us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.005091 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.006320 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.006621 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.006721 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.009614 (+ 2893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:05.009618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:05.009620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:05.009650 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:05.011125 (+ 1475us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.011801 (+ 676us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.011804 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.011805 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.011906 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.012288 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.012291 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.012293 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.012510 (+ 217us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:05.012604 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:05.013270 (+ 666us) server_negotiation.cc:299] Negotiation successful
0807 00:02:05.013437 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:02:05.014438 5051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.000580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37565 (local address 127.30.204.65:56039)
0807 00:02:05.001142 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:05.001196 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:05.002627 (+ 1431us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:05.003295 (+ 668us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:05.003304 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:05.003793 (+ 489us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:05.004360 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:05.004376 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.005230 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.005233 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:05.006178 (+ 945us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:05.006189 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.006817 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.006822 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:05.007040 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.007693 (+ 653us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:05.007713 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:05.009409 (+ 1696us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:05.011271 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:05.011279 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:05.011301 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:05.011687 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:05.011999 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:05.012003 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:05.012005 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:05.012179 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:05.012623 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:05.012632 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:05.012933 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 00:02:05.013129 (+ 196us) client_negotiation.cc:240] Negotiation successful
0807 00:02:05.013485 (+ 356us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":332,"thread_start_us":138,"threads_started":1}
I20260807 00:02:05.015882 5049 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37565
I20260807 00:02:05.016268 5049 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:05.017060 5049 heartbeater.cc:511] Master 127.30.204.126:37565 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:05.019137 4744 ts_manager.cc:194] Registered new tserver with Master: 29dbbc480257431db465fd7d1eea4cbb (127.30.204.65:45683)
I20260807 00:02:05.020511 4744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:56039
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:02:05.040096 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37565
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:05.165020 5055 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:05.165347 5055 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:05.165395 5055 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:05.169775 5055 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:05.169859 5055 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:05.169924 5055 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:05.175815 5055 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5055
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:05.177075 5055 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:05.178257 5055 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:05.189355 5062 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:05.189355 5066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:05.189355 5064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:05.189924 5055 server_base.cc:1034] running on GCE node
I20260807 00:02:05.190442 5055 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:05.191195 5055 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:05.192414 5055 hybrid_clock.cc:648] HybridClock initialized: now 1786060925192382 us; error 48 us; skew 500 ppm
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:05.195524 5055 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:02:05.196583 5055 webserver.cc:460] Webserver started at http://127.30.204.66:37207/ using document root <none> and password file <none>
I20260807 00:02:05.197192 5055 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:05.197269 5055 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:05.197494 5055 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:05.199800 5055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "5e4a82b4c3954cc29dab717fd92ab91e"
format_stamp: "Formatted at 2026-08-07 00:02:05 on dist-test-slave-pww0"
server_key: "0934bfcd4dcdd69cc06c53294bd810aa"
server_key_iv: "38b6e867a449229c843d7e7a09fc34f3"
server_key_version: "encryptionkey@0"
I20260807 00:02:05.200415 5055 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "5e4a82b4c3954cc29dab717fd92ab91e"
format_stamp: "Formatted at 2026-08-07 00:02:05 on dist-test-slave-pww0"
server_key: "0934bfcd4dcdd69cc06c53294bd810aa"
server_key_iv: "38b6e867a449229c843d7e7a09fc34f3"
server_key_version: "encryptionkey@0"
I20260807 00:02:05.204700 5055 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:05.207762 5080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:05.209297 5055 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:05.209462 5055 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "5e4a82b4c3954cc29dab717fd92ab91e"
format_stamp: "Formatted at 2026-08-07 00:02:05 on dist-test-slave-pww0"
server_key: "0934bfcd4dcdd69cc06c53294bd810aa"
server_key_iv: "38b6e867a449229c843d7e7a09fc34f3"
server_key_version: "encryptionkey@0"
I20260807 00:02:05.209596 5055 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:05.229061 5055 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:05.232676 5055 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:05.232889 5055 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:05.233592 5055 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:05.234552 5055 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:05.234607 5055 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:05.234668 5055 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:05.234695 5055 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:05.243992 5055 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:37067
I20260807 00:02:05.244014 5243 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:37067 every 8 connection(s)
I20260807 00:02:05.245007 5055 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:05.248571 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 5055
I20260807 00:02:05.248693 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:05.248948 31537 external_mini_cluster.cc:1428] Setting key 231e95e767e7fcb6ea46790361f23a80
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:02:05.260706 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.247053 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:46035 (local address 127.30.204.126:37565)
0807 00:02:05.247190 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:05.247194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:05.248315 (+ 1121us) server_negotiation.cc:407] Connection header received
0807 00:02:05.249873 (+ 1558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:05.249876 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:05.249936 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:05.250012 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:05.251387 (+ 1375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.251973 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.253423 (+ 1450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.253664 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.253769 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.256611 (+ 2842us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:05.256614 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:05.256616 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:05.256649 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:05.258196 (+ 1547us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.258853 (+ 657us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.258857 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.258858 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.258949 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.259373 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.259377 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.259379 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.259640 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:05.259737 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:05.260371 (+ 634us) server_negotiation.cc:299] Negotiation successful
0807 00:02:05.260526 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:02:05.261792 5247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.247521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37565 (local address 127.30.204.66:46035)
0807 00:02:05.248104 (+ 583us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:05.248157 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:05.249627 (+ 1470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:05.250234 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:05.250243 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:05.250663 (+ 420us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:05.251212 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:05.251229 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.252124 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.252127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:05.253273 (+ 1146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:05.253288 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.253840 (+ 552us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.253845 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:05.254044 (+ 199us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.254744 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:05.254764 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:05.256394 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:05.258359 (+ 1965us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:05.258368 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:05.258389 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:05.258736 (+ 347us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:05.259058 (+ 322us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:05.259061 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:05.259064 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:05.259275 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:05.259755 (+ 480us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:05.259763 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:05.260086 (+ 323us) client_negotiation.cc:769] Sending connection context
0807 00:02:05.260311 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 00:02:05.260739 (+ 428us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":132,"threads_started":1}
I20260807 00:02:05.263424 5244 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37565
I20260807 00:02:05.263753 5244 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:05.264397 5244 heartbeater.cc:511] Master 127.30.204.126:37565 requested a full tablet report, sending...
I20260807 00:02:05.265578 4744 ts_manager.cc:194] Registered new tserver with Master: 5e4a82b4c3954cc29dab717fd92ab91e (127.30.204.66:37067)
I20260807 00:02:05.266186 4744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:46035
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:02:05.279249 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37565
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:39093
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:05.401171 5254 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:05.401481 5254 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:05.401525 5254 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:05.405731 5254 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:05.405818 5254 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:05.405879 5254 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:05.411572 5254 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:39093
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37565
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5254
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:05.412776 5254 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:05.413949 5254 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:05.424615 5261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:05.424860 5254 server_base.cc:1034] running on GCE node
W20260807 00:02:05.424609 5263 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:05.424615 5266 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:05.425391 5254 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:05.426113 5254 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:05.427316 5254 hybrid_clock.cc:648] HybridClock initialized: now 1786060925427281 us; error 46 us; skew 500 ppm
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:05.430401 5254 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:02:05.431445 5254 webserver.cc:460] Webserver started at http://127.30.204.67:33805/ using document root <none> and password file <none>
I20260807 00:02:05.432107 5254 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:05.432186 5254 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:05.432461 5254 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:05.434805 5254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "107abfbef5e34103b47e4bc1157d5edf"
format_stamp: "Formatted at 2026-08-07 00:02:05 on dist-test-slave-pww0"
server_key: "1104b4750784328d48b4618e92a589e6"
server_key_iv: "c83f75e1ffaf2776321d346aa62e7b11"
server_key_version: "encryptionkey@0"
I20260807 00:02:05.435441 5254 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "107abfbef5e34103b47e4bc1157d5edf"
format_stamp: "Formatted at 2026-08-07 00:02:05 on dist-test-slave-pww0"
server_key: "1104b4750784328d48b4618e92a589e6"
server_key_iv: "c83f75e1ffaf2776321d346aa62e7b11"
server_key_version: "encryptionkey@0"
I20260807 00:02:05.439620 5254 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 00:02:05.442500 5273 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:05.443974 5254 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:05.444089 5254 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "107abfbef5e34103b47e4bc1157d5edf"
format_stamp: "Formatted at 2026-08-07 00:02:05 on dist-test-slave-pww0"
server_key: "1104b4750784328d48b4618e92a589e6"
server_key_iv: "c83f75e1ffaf2776321d346aa62e7b11"
server_key_version: "encryptionkey@0"
I20260807 00:02:05.444197 5254 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:05.461968 5254 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:05.465528 5254 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:05.465716 5254 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:05.466323 5254 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:05.467227 5254 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:05.467293 5254 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:05.467338 5254 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:05.467355 5254 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:05.476826 5254 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:46631
I20260807 00:02:05.476850 5443 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:46631 every 8 connection(s)
I20260807 00:02:05.477842 5254 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:05.487659 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 5254
I20260807 00:02:05.487833 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060925, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:05.488173 31537 external_mini_cluster.cc:1428] Setting key 3b2e9e5f2dae18a7629e4ba4b88fa3cc
I20260807 00:02:05.492645 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.479803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:57179 (local address 127.30.204.126:37565)
0807 00:02:05.479947 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:05.479951 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:05.480866 (+ 915us) server_negotiation.cc:407] Connection header received
0807 00:02:05.482324 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:05.482328 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:05.482396 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:05.482474 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:05.483843 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.484382 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.485579 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.485813 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.485907 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.488779 (+ 2872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:05.488783 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:05.488785 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:05.488820 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:05.490470 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.491010 (+ 540us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.491014 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.491016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.491109 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.491472 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.491476 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.491478 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.491685 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:05.491779 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:05.492331 (+ 552us) server_negotiation.cc:299] Negotiation successful
0807 00:02:05.492475 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:02:05.493563 5446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.480172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37565 (local address 127.30.204.67:57179)
0807 00:02:05.480691 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:05.480744 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:05.482061 (+ 1317us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:05.482685 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:05.482693 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:05.483092 (+ 399us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:05.483682 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:05.483701 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.484520 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.484523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:05.485445 (+ 922us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:05.485456 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.485976 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.485980 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:05.486153 (+ 173us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.486762 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:05.486780 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:05.488584 (+ 1804us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:05.490587 (+ 2003us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:05.490594 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:05.490612 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:05.490910 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:05.491195 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:05.491198 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:05.491200 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:05.491361 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:05.491793 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:05.491801 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:05.492080 (+ 279us) client_negotiation.cc:769] Sending connection context
0807 00:02:05.492281 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 00:02:05.492668 (+ 387us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":124,"threads_started":1}
I20260807 00:02:05.494805 5444 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37565
I20260807 00:02:05.495035 5444 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:05.495579 5444 heartbeater.cc:511] Master 127.30.204.126:37565 requested a full tablet report, sending...
I20260807 00:02:05.496709 4744 ts_manager.cc:194] Registered new tserver with Master: 107abfbef5e34103b47e4bc1157d5edf (127.30.204.67:46631)
I20260807 00:02:05.497269 4744 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:57179
I20260807 00:02:05.502705 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:02:05.532384 5452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.521983 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50798 (local address 127.30.204.65:45683)
0807 00:02:05.522296 (+ 313us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:05.522305 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:05.522336 (+ 31us) server_negotiation.cc:407] Connection header received
0807 00:02:05.522390 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:05.522395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:05.522726 (+ 331us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:05.522883 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:05.523788 (+ 905us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.524607 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.525545 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.525987 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.526090 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.528789 (+ 2699us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:05.528804 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:05.528821 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:05.528856 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:05.530550 (+ 1694us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.531018 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.531024 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.531025 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.531116 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.531364 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.531366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.531367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.531687 (+ 320us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:05.531864 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:05.532084 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 00:02:05.532237 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":100,"threads_started":1}
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[4670](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060925, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:05.544021 4804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:05.535548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54228 (local address 127.30.204.126:37565)
0807 00:02:05.535711 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:05.535715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:05.535729 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:05.535880 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:05.535883 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:05.535932 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:05.536004 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:05.536592 (+ 588us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.537128 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.538029 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:05.538292 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:05.538397 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:05.540892 (+ 2495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:05.540895 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:05.540897 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:05.540930 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:05.542286 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.542708 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.542711 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.542713 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.542803 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:05.543061 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:05.543064 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:05.543066 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:05.543318 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:05.543449 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:05.543722 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 00:02:05.543855 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
W20260807 00:02:05.544499 4744 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:54228
I20260807 00:02:05.545224 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 4841
I20260807 00:02:05.564669 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 5055
I20260807 00:02:05.575860 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 5254
I20260807 00:02:05.617020 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 4693
2026-08-07T00:02:05Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2406 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[5458](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[5458](info): set up 2 sockets
Aug 07 00:02:05 dist-test-slave-pww0 krb5kdc[5458](info): commencing operation
krb5kdc: starting...
W20260807 00:02:06.794883 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.141s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:06 dist-test-slave-pww0 krb5kdc[5458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:06Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:06.928647 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:44317
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:44317
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:07.050650 5474 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:07.050992 5474 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:07.051039 5474 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:07.055325 5474 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:07.055413 5474 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:07.055433 5474 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:07.055449 5474 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:07.055500 5474 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:07.061336 5474 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:44317
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:44317
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5474
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:07.062547 5474 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:07.063685 5474 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:07.072661 5480 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:07.072656 5485 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:07.072733 5474 server_base.cc:1034] running on GCE node
W20260807 00:02:07.072656 5482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:07.073640 5474 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:07.074898 5474 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:07.076102 5474 hybrid_clock.cc:648] HybridClock initialized: now 1786060927076072 us; error 45 us; skew 500 ppm
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:07.079524 5474 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:07.080461 5474 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:07.080587 5474 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:07.080839 5474 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:07.083333 5474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "7ed1c05775a0439fb5a33ffb4ee27c19"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "09c49b3df5daa9b3583688945a7484ac"
server_key_iv: "51a1c9b29e4ede2deb2fbf94263cecc4"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.084112 5474 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "7ed1c05775a0439fb5a33ffb4ee27c19"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "09c49b3df5daa9b3583688945a7484ac"
server_key_iv: "51a1c9b29e4ede2deb2fbf94263cecc4"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.088737 5474 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:02:07.091936 5497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.093446 5474 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:07.093611 5474 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "7ed1c05775a0439fb5a33ffb4ee27c19"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "09c49b3df5daa9b3583688945a7484ac"
server_key_iv: "51a1c9b29e4ede2deb2fbf94263cecc4"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.093743 5474 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:07.113914 5474 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:07.117796 5474 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:07.118018 5474 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:07.126065 5474 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:44317
I20260807 00:02:07.126399 5570 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:44317 every 8 connection(s)
I20260807 00:02:07.126993 5474 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:07.131035 5571 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:07.136987 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 5474
I20260807 00:02:07.137125 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:07.137424 31537 external_mini_cluster.cc:1428] Setting key 23eeb117dff08399721ca2be705eae86
I20260807 00:02:07.138741 5571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19: Bootstrap starting.
I20260807 00:02:07.141378 5571 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:07.142236 5571 log.cc:824] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:07.144556 5571 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19: No bootstrap required, opened a new log
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060926, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:07.149135 5571 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44317 } }
I20260807 00:02:07.149432 5571 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:07.149470 5571 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7ed1c05775a0439fb5a33ffb4ee27c19, State: Initialized, Role: FOLLOWER
I20260807 00:02:07.150028 5571 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44317 } }
I20260807 00:02:07.150228 5571 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:07.150362 5571 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:07.150485 5571 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:07.151639 5571 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44317 } }
I20260807 00:02:07.152091 5571 leader_election.cc:302] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7ed1c05775a0439fb5a33ffb4ee27c19; no voters:
I20260807 00:02:07.152554 5571 leader_election.cc:288] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:07.152699 5576 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:07.152956 5576 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [term 1 LEADER]: Becoming Leader. State: Replica: 7ed1c05775a0439fb5a33ffb4ee27c19, State: Running, Role: LEADER
I20260807 00:02:07.153375 5576 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44317 } }
I20260807 00:02:07.153890 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.139088 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51124 (local address 127.30.204.126:44317)
0807 00:02:07.139593 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.139611 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.139659 (+ 48us) server_negotiation.cc:407] Connection header received
0807 00:02:07.140375 (+ 716us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.140395 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.140782 (+ 387us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.141138 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:07.142005 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.143067 (+ 1062us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.144055 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.144567 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.144750 (+ 183us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.147783 (+ 3033us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:07.147790 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:07.147809 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:07.147850 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:07.149705 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.150343 (+ 638us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.150350 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.150358 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.150534 (+ 176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.150884 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.150887 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.150888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.151403 (+ 515us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:07.151637 (+ 234us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.152375 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.152778 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":314,"thread_start_us":129,"threads_started":1}
I20260807 00:02:07.154019 5571 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:07.155977 5577 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44317 } } }
I20260807 00:02:07.156139 5577 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:07.156466 5579 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7ed1c05775a0439fb5a33ffb4ee27c19. Latest consensus state: current_term: 1 leader_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7ed1c05775a0439fb5a33ffb4ee27c19" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44317 } } }
I20260807 00:02:07.159587 5579 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19 [sys.catalog]: This master's current role is: LEADER
W20260807 00:02:07.159914 5597 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:02:07.159986 5597 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:02:07.160050 5594 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:07.160861 5594 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:07.167536 5594 catalog_manager.cc:1347] Generated new cluster ID: 38c8b77095fc410ab381446cdc54b126
I20260807 00:02:07.167753 5594 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:07.187284 5594 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:07.188431 5594 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:07.198036 5594 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19: Generated new TSK 0
I20260807 00:02:07.198840 5594 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:02:07.234760 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44317
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:07.361924 5606 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:07.362265 5606 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:07.362334 5606 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:07.366717 5606 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:07.366815 5606 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:07.366906 5606 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:07.373382 5606 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5606
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:07.374696 5606 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:07.376106 5606 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:07.386868 5612 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:07.386878 5611 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:07.386895 5618 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:07.387422 5606 server_base.cc:1034] running on GCE node
I20260807 00:02:07.388000 5606 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:07.388769 5606 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:07.389969 5606 hybrid_clock.cc:648] HybridClock initialized: now 1786060927389929 us; error 55 us; skew 500 ppm
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:07.393296 5606 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:02:07.394172 5606 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:07.394254 5606 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:07.394450 5606 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:07.396816 5606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "88c724a94d994f8692b6ff4933f7f542"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "c3cd4282986056ed7de2d8969b3d7a24"
server_key_iv: "43851e059919a8fd0afc086bdfccd716"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.397508 5606 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "88c724a94d994f8692b6ff4933f7f542"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "c3cd4282986056ed7de2d8969b3d7a24"
server_key_iv: "43851e059919a8fd0afc086bdfccd716"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.401948 5606 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:07.405109 5625 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.406628 5606 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:07.406764 5606 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "88c724a94d994f8692b6ff4933f7f542"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "c3cd4282986056ed7de2d8969b3d7a24"
server_key_iv: "43851e059919a8fd0afc086bdfccd716"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.406876 5606 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:07.421391 5606 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:07.425010 5606 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:07.425210 5606 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:07.425827 5606 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:07.426745 5606 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:07.426797 5606 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.426841 5606 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:07.426862 5606 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.436647 5606 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:45947
I20260807 00:02:07.436669 5799 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:45947 every 8 connection(s)
I20260807 00:02:07.437364 5606 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:07.443939 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 5606
I20260807 00:02:07.444092 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:07.444438 31537 external_mini_cluster.cc:1428] Setting key e9e768a8b24a7cc757c8f2bcb117500e
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:07.452571 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.439339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:41339 (local address 127.30.204.126:44317)
0807 00:02:07.439595 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.439599 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.440831 (+ 1232us) server_negotiation.cc:407] Connection header received
0807 00:02:07.441736 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.441739 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.441794 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.441878 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:07.443237 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.443919 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.445408 (+ 1489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.445753 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.445854 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.448617 (+ 2763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:07.448621 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:07.448624 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:07.448653 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:07.450009 (+ 1356us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.450674 (+ 665us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.450677 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.450679 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.450764 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.451241 (+ 477us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.451244 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.451245 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.451499 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:07.451601 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.452314 (+ 713us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.452436 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":77}
I20260807 00:02:07.453958 5802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.439811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:44317 (local address 127.30.204.65:41339)
0807 00:02:07.440625 (+ 814us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:07.440676 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:07.441504 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:07.442047 (+ 543us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:07.442054 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:07.442445 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:07.443082 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.443099 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.444078 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.444082 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.445291 (+ 1209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.445304 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.445930 (+ 626us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.445935 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.446104 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.446724 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:07.446743 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:07.448404 (+ 1661us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:07.450144 (+ 1740us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:07.450151 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:07.450170 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:07.450557 (+ 387us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:07.450854 (+ 297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:07.450858 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:07.450860 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:07.451098 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:07.451622 (+ 524us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:07.451630 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:07.452024 (+ 394us) client_negotiation.cc:769] Sending connection context
0807 00:02:07.452282 (+ 258us) client_negotiation.cc:240] Negotiation successful
0807 00:02:07.452703 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":571,"thread_start_us":133,"threads_started":1}
I20260807 00:02:07.455497 5800 heartbeater.cc:347] Connected to a master server at 127.30.204.126:44317
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:07.455797 5800 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:07.456313 5800 heartbeater.cc:511] Master 127.30.204.126:44317 requested a full tablet report, sending...
I20260807 00:02:07.458204 5520 ts_manager.cc:194] Registered new tserver with Master: 88c724a94d994f8692b6ff4933f7f542 (127.30.204.65:45947)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:02:07.459601 5520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:41339
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:02:07.478421 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44317
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:07.602609 5807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:07.602948 5807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:07.603017 5807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:07.607308 5807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:07.607403 5807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:07.607530 5807 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:07.613440 5807 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.5807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:07.614739 5807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:07.615931 5807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:07.626794 5813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:07.626919 5807 server_base.cc:1034] running on GCE node
W20260807 00:02:07.626814 5814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:07.626794 5818 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:07.627647 5807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:07.628391 5807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:07.629606 5807 hybrid_clock.cc:648] HybridClock initialized: now 1786060927629578 us; error 46 us; skew 500 ppm
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:07.632793 5807 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:02:07.633659 5807 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:07.633742 5807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:07.633961 5807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:07.636329 5807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "ab53145cf40647a98bc296fb1fe3ffcd"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "1c776be11bcb0567b1d6c6f6915cafe3"
server_key_iv: "de12155e5daae1c3330db5fa706f7b28"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.636971 5807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "ab53145cf40647a98bc296fb1fe3ffcd"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "1c776be11bcb0567b1d6c6f6915cafe3"
server_key_iv: "de12155e5daae1c3330db5fa706f7b28"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.641357 5807 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:07.644316 5828 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.645932 5807 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:07.646100 5807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "ab53145cf40647a98bc296fb1fe3ffcd"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "1c776be11bcb0567b1d6c6f6915cafe3"
server_key_iv: "de12155e5daae1c3330db5fa706f7b28"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.646281 5807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:07.660420 5807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:07.664099 5807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:07.664307 5807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:07.664976 5807 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:07.665956 5807 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:07.666006 5807 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.666049 5807 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:07.666065 5807 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.675140 5807 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:41175
I20260807 00:02:07.675170 5991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:41175 every 8 connection(s)
I20260807 00:02:07.675889 5807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:07.677018 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 5807
I20260807 00:02:07.677146 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:07.677476 31537 external_mini_cluster.cc:1428] Setting key 365d41cb31e12f4d9bfcecdcbb7685c9
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:02:07.692238 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.678047 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:41917 (local address 127.30.204.126:44317)
0807 00:02:07.678198 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.678203 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.679363 (+ 1160us) server_negotiation.cc:407] Connection header received
0807 00:02:07.680889 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.680893 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.680951 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.681041 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:07.682903 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.683498 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.684735 (+ 1237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.684970 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.685075 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.688035 (+ 2960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:07.688039 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:07.688041 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:07.688073 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:07.689652 (+ 1579us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.690249 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.690251 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.690253 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.690374 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.690866 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.690869 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.690870 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.691133 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:07.691252 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.691908 (+ 656us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.692057 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:02:07.693166 5997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.678578 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:44317 (local address 127.30.204.66:41917)
0807 00:02:07.679195 (+ 617us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:07.679257 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:07.680650 (+ 1393us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:07.681336 (+ 686us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:07.681347 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:07.681936 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:07.682710 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.682735 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.683653 (+ 918us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.683657 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.684595 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.684607 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.685134 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.685138 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.685332 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.686013 (+ 681us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:07.686031 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:07.687775 (+ 1744us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:07.689800 (+ 2025us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:07.689806 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:07.689823 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:07.690120 (+ 297us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:07.690486 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:07.690488 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:07.690490 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:07.690722 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:07.691273 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:07.691281 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:07.691639 (+ 358us) client_negotiation.cc:769] Sending connection context
0807 00:02:07.691840 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 00:02:07.692238 (+ 398us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":368,"thread_start_us":180,"threads_started":1}
I20260807 00:02:07.694674 5992 heartbeater.cc:347] Connected to a master server at 127.30.204.126:44317
I20260807 00:02:07.694979 5992 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:07.695657 5992 heartbeater.cc:511] Master 127.30.204.126:44317 requested a full tablet report, sending...
I20260807 00:02:07.696998 5520 ts_manager.cc:194] Registered new tserver with Master: ab53145cf40647a98bc296fb1fe3ffcd (127.30.204.66:41175)
I20260807 00:02:07.697763 5520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:41917
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:02:07.710831 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44317
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36879
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:07.832173 6001 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:07.832484 6001 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:07.832531 6001 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:07.836741 6001 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:07.836826 6001 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:07.836894 6001 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:07.842720 6001 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36879
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44317
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6001
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:07.843940 6001 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:07.845037 6001 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:07.855525 6009 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:07.855549 6010 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:07.855757 6012 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:07.855779 6001 server_base.cc:1034] running on GCE node
I20260807 00:02:07.856333 6001 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:07.857014 6001 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:07.858208 6001 hybrid_clock.cc:648] HybridClock initialized: now 1786060927858184 us; error 39 us; skew 500 ppm
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:07.861371 6001 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:02:07.862231 6001 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:07.862314 6001 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:07.862540 6001 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:07.864820 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "ada3e53d883a8751ed4f98089ce02c3b"
server_key_iv: "98067e33e311f9203c09b78f03e5eceb"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.865453 6001 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "ada3e53d883a8751ed4f98089ce02c3b"
server_key_iv: "98067e33e311f9203c09b78f03e5eceb"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.869623 6001 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:07.872723 6019 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.874348 6001 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:02:07.874506 6001 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa"
format_stamp: "Formatted at 2026-08-07 00:02:07 on dist-test-slave-pww0"
server_key: "ada3e53d883a8751ed4f98089ce02c3b"
server_key_iv: "98067e33e311f9203c09b78f03e5eceb"
server_key_version: "encryptionkey@0"
I20260807 00:02:07.874639 6001 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:07.905898 6001 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:07.909559 6001 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:07.909770 6001 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:07.910488 6001 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:07.911494 6001 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:07.911556 6001 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.911615 6001 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:07.911641 6001 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:07.922338 6001 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:41999
I20260807 00:02:07.922362 6185 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:41999 every 8 connection(s)
I20260807 00:02:07.923094 6001 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:07.929668 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 6001
I20260807 00:02:07.929816 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:07.930145 31537 external_mini_cluster.cc:1428] Setting key 8789cf17a210ad7bc765b222b6ca0611
Aug 07 00:02:07 dist-test-slave-pww0 krb5kdc[5458](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060927, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:07.938155 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.925112 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:35811 (local address 127.30.204.126:44317)
0807 00:02:07.925299 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.925304 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.926303 (+ 999us) server_negotiation.cc:407] Connection header received
0807 00:02:07.927761 (+ 1458us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.927764 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.927821 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.927911 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:07.929197 (+ 1286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.929972 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.931623 (+ 1651us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.931874 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.931959 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.934681 (+ 2722us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:07.934684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:07.934686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:07.934715 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:07.935964 (+ 1249us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.936515 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.936518 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.936520 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.936604 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.936955 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.936959 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.936961 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.937169 (+ 208us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:07.937271 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.937826 (+ 555us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.937977 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 00:02:07.939386 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.925541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:44317 (local address 127.30.204.67:35811)
0807 00:02:07.926119 (+ 578us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:07.926171 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:07.927440 (+ 1269us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:07.928089 (+ 649us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:07.928099 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:07.928496 (+ 397us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:07.929059 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.929073 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.930151 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.930155 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.931362 (+ 1207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.931375 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.932041 (+ 666us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.932045 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.932274 (+ 229us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.932937 (+ 663us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:07.932956 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:07.934489 (+ 1533us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:07.936086 (+ 1597us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:07.936094 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:07.936114 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:07.936407 (+ 293us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:07.936690 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:07.936694 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:07.936696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:07.936861 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:07.937275 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:07.937283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:07.937573 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 00:02:07.937770 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 00:02:07.938164 (+ 394us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":143,"threads_started":1}
I20260807 00:02:07.940845 6186 heartbeater.cc:347] Connected to a master server at 127.30.204.126:44317
I20260807 00:02:07.941156 6186 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:07.941702 6186 heartbeater.cc:511] Master 127.30.204.126:44317 requested a full tablet report, sending...
I20260807 00:02:07.942972 5520 ts_manager.cc:194] Registered new tserver with Master: 6a83110be7bd4dd8b1dd9b8516cfdeaa (127.30.204.67:41999)
I20260807 00:02:07.943742 5520 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:35811
I20260807 00:02:07.944783 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:02:07.954890 5574 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.946864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51130 (local address 127.30.204.126:44317)
0807 00:02:07.947090 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.947096 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.947228 (+ 132us) server_negotiation.cc:407] Connection header received
0807 00:02:07.947388 (+ 160us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.947391 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.947439 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.947548 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:07.948305 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.948848 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.949721 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.949959 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.950053 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.951589 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:07.951594 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:07.951596 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:07.951629 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:07.953046 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.953504 (+ 458us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.953507 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.953508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.953596 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:07.953878 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:07.953881 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:07.953883 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:07.954155 (+ 272us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:07.954284 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.954553 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.954685 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":123}
I20260807 00:02:07.958693 5520 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51130:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:07.962073 5520 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:07.978703 6209 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.972002 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:41999 (local address 127.0.0.1:34224)
0807 00:02:07.972860 (+ 858us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:07.972923 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:07.973077 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:07.974083 (+ 1006us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:07.974086 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:07.974109 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:07.974333 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.974342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.975970 (+ 1628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.975973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.977141 (+ 1168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.977152 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.978060 (+ 908us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.978064 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.978204 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.978255 (+ 51us) client_negotiation.cc:769] Sending connection context
0807 00:02:07.978379 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 00:02:07.978528 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":698,"mutex_wait_us":56,"spinlock_wait_cycles":15232,"thread_start_us":136,"threads_started":1}
I20260807 00:02:07.978706 6211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.972830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34224 (local address 127.30.204.67:41999)
0807 00:02:07.973455 (+ 625us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.973467 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.973485 (+ 18us) server_negotiation.cc:407] Connection header received
0807 00:02:07.973555 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.973560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.973763 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.973921 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:07.974487 (+ 566us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.975767 (+ 1280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.977286 (+ 1519us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.977930 (+ 644us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.978039 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.978278 (+ 239us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.978416 (+ 138us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.978528 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":529,"thread_start_us":107,"threads_started":1}
I20260807 00:02:07.979051 6206 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.972254 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:45947 (local address 127.0.0.1:38678)
0807 00:02:07.973295 (+ 1041us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:07.973308 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:07.973391 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:07.974083 (+ 692us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:07.974087 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:07.974115 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:07.974387 (+ 272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.974392 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.975422 (+ 1030us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.975427 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.976982 (+ 1555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.976995 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.978473 (+ 1478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.978477 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.978687 (+ 210us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.978743 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 00:02:07.978813 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 00:02:07.978897 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":988,"mutex_wait_us":336,"spinlock_wait_cycles":5248,"thread_start_us":84,"threads_started":1}
I20260807 00:02:07.979542 6210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.972688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38678 (local address 127.30.204.65:45947)
0807 00:02:07.973264 (+ 576us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.973273 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.973475 (+ 202us) server_negotiation.cc:407] Connection header received
0807 00:02:07.973555 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.973560 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.973771 (+ 211us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.973921 (+ 150us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:07.974514 (+ 593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.975293 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.977138 (+ 1845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.978119 (+ 981us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.978259 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.978418 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.979214 (+ 796us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.979338 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":116,"threads_started":1}
I20260807 00:02:07.979784 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.971952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:41175 (local address 127.0.0.1:59572)
0807 00:02:07.972861 (+ 909us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:07.972923 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:07.973077 (+ 154us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:07.973977 (+ 900us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:07.973983 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:07.974006 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:07.974264 (+ 258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.974269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.975918 (+ 1649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.975922 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.977213 (+ 1291us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:07.977224 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.978463 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.978467 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:07.978684 (+ 217us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.978743 (+ 59us) client_negotiation.cc:769] Sending connection context
0807 00:02:07.979390 (+ 647us) client_negotiation.cc:240] Negotiation successful
0807 00:02:07.979498 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":718,"mutex_wait_us":198,"spinlock_wait_cycles":6144,"thread_start_us":76,"threads_started":1}
I20260807 00:02:07.981925 6082 tablet_service.cc:1467] Processing CreateTablet for tablet cbba473a6b8c4f12ab49278e8073d810 (DEFAULT_TABLE table=test-table [id=a3155f564a5f466397554312d5b67d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:07.981953 5698 tablet_service.cc:1467] Processing CreateTablet for tablet cbba473a6b8c4f12ab49278e8073d810 (DEFAULT_TABLE table=test-table [id=a3155f564a5f466397554312d5b67d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:07.983003 5698 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbba473a6b8c4f12ab49278e8073d810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:07.983014 6082 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbba473a6b8c4f12ab49278e8073d810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:07.988178 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:07.972089 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59572 (local address 127.30.204.66:41175)
0807 00:02:07.973399 (+ 1310us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:07.973412 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:07.973439 (+ 27us) server_negotiation.cc:407] Connection header received
0807 00:02:07.973518 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:07.973523 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:07.973776 (+ 253us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:07.974399 (+ 623us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:07.974573 (+ 174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.975767 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.977324 (+ 1557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:07.978318 (+ 994us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:07.980175 (+ 1857us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:07.987783 (+ 7608us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:07.987865 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 00:02:07.987991 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1185,"thread_start_us":190,"threads_started":1}
I20260807 00:02:07.990332 6215 tablet_bootstrap.cc:492] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Bootstrap starting.
I20260807 00:02:07.990810 6212 tablet_bootstrap.cc:492] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: Bootstrap starting.
I20260807 00:02:07.991626 5896 tablet_service.cc:1467] Processing CreateTablet for tablet cbba473a6b8c4f12ab49278e8073d810 (DEFAULT_TABLE table=test-table [id=a3155f564a5f466397554312d5b67d08]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:07.992690 5896 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cbba473a6b8c4f12ab49278e8073d810. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:07.993415 6212 tablet_bootstrap.cc:654] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:07.993683 6215 tablet_bootstrap.cc:654] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:07.994418 6212 log.cc:824] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:07.995040 6215 log.cc:824] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:07.997016 6212 tablet_bootstrap.cc:492] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: No bootstrap required, opened a new log
I20260807 00:02:07.997234 6212 ts_tablet_manager.cc:1397] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 00:02:07.999503 6215 tablet_bootstrap.cc:492] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: No bootstrap required, opened a new log
I20260807 00:02:07.999732 6221 tablet_bootstrap.cc:492] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Bootstrap starting.
I20260807 00:02:07.999943 6215 ts_tablet_manager.cc:1397] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Time spent bootstrapping tablet: real 0.010s user 0.006s sys 0.000s
I20260807 00:02:08.002471 6221 tablet_bootstrap.cc:654] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:08.002302 6212 raft_consensus.cc:348] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.002640 6212 raft_consensus.cc:374] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:08.002697 6212 raft_consensus.cc:729] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88c724a94d994f8692b6ff4933f7f542, State: Initialized, Role: FOLLOWER
I20260807 00:02:08.003232 6212 consensus_queue.cc:260] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.003608 6221 log.cc:824] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:08.004221 5800 heartbeater.cc:503] Master 127.30.204.126:44317 was elected leader, sending a full tablet report...
I20260807 00:02:08.004601 6212 ts_tablet_manager.cc:1428] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 00:02:08.005199 6215 raft_consensus.cc:348] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.005538 6215 raft_consensus.cc:374] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:08.005620 6215 raft_consensus.cc:729] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6a83110be7bd4dd8b1dd9b8516cfdeaa, State: Initialized, Role: FOLLOWER
I20260807 00:02:08.006120 6215 consensus_queue.cc:260] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.006810 6221 tablet_bootstrap.cc:492] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: No bootstrap required, opened a new log
I20260807 00:02:08.006930 6186 heartbeater.cc:503] Master 127.30.204.126:44317 was elected leader, sending a full tablet report...
I20260807 00:02:08.007051 6221 ts_tablet_manager.cc:1397] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Time spent bootstrapping tablet: real 0.008s user 0.005s sys 0.000s
I20260807 00:02:08.007360 6215 ts_tablet_manager.cc:1428] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
I20260807 00:02:08.010861 6221 raft_consensus.cc:348] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.011202 6221 raft_consensus.cc:374] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:08.011272 6221 raft_consensus.cc:729] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab53145cf40647a98bc296fb1fe3ffcd, State: Initialized, Role: FOLLOWER
I20260807 00:02:08.011826 6221 consensus_queue.cc:260] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.012583 5992 heartbeater.cc:503] Master 127.30.204.126:44317 was elected leader, sending a full tablet report...
I20260807 00:02:08.012987 6221 ts_tablet_manager.cc:1428] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:02:08.159852 6225 raft_consensus.cc:482] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:08.160132 6225 raft_consensus.cc:504] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.161399 6225 leader_election.cc:288] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers ab53145cf40647a98bc296fb1fe3ffcd (127.30.204.66:41175), 88c724a94d994f8692b6ff4933f7f542 (127.30.204.65:45947)
I20260807 00:02:08.166539 6208 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.161870 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:35987 (local address 127.30.204.66:41175)
0807 00:02:08.162207 (+ 337us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:08.162212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:08.162226 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:08.162279 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:08.162281 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:08.162326 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:08.162393 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:08.162793 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.163563 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.165200 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.165831 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.165966 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:08.166021 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:08.166324 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 00:02:08.166399 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":245}
I20260807 00:02:08.166539 6210 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.162217 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:36403 (local address 127.30.204.65:45947)
0807 00:02:08.162350 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:08.162355 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:08.162373 (+ 18us) server_negotiation.cc:407] Connection header received
0807 00:02:08.162600 (+ 227us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:08.162603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:08.162652 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:08.162735 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:08.163035 (+ 300us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.163749 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.165201 (+ 1452us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.165851 (+ 650us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.165966 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:08.166021 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:08.166324 (+ 303us) server_negotiation.cc:299] Negotiation successful
0807 00:02:08.166399 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:02:08.166617 6188 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.161695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:41175 (local address 127.30.204.67:35987)
0807 00:02:08.161915 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:08.161932 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:08.162025 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:08.162522 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:08.162524 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:08.162543 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:08.162667 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:08.162674 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.163689 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.163692 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:08.165049 (+ 1357us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:08.165061 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.165985 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.165989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:08.166207 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:08.166230 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:02:08.166309 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 00:02:08.166454 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":82}
I20260807 00:02:08.166839 6228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.161756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:45947 (local address 127.30.204.67:36403)
0807 00:02:08.162268 (+ 512us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:08.162284 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:08.162405 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:08.162767 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:08.162770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:08.162792 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:08.162921 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:08.162927 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.163842 (+ 915us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.163844 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:08.165050 (+ 1206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:08.165061 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.165985 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.165989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:08.166144 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:08.166222 (+ 78us) client_negotiation.cc:769] Sending connection context
0807 00:02:08.166309 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:02:08.166454 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":435,"mutex_wait_us":128,"thread_start_us":154,"threads_started":1}
I20260807 00:02:08.167367 5928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba473a6b8c4f12ab49278e8073d810" candidate_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" is_pre_election: true
I20260807 00:02:08.167486 5723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba473a6b8c4f12ab49278e8073d810" candidate_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c724a94d994f8692b6ff4933f7f542" is_pre_election: true
I20260807 00:02:08.167794 5723 raft_consensus.cc:2454] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a83110be7bd4dd8b1dd9b8516cfdeaa in term 0.
I20260807 00:02:08.167794 5928 raft_consensus.cc:2454] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6a83110be7bd4dd8b1dd9b8516cfdeaa in term 0.
I20260807 00:02:08.168356 6020 leader_election.cc:302] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 6a83110be7bd4dd8b1dd9b8516cfdeaa, 88c724a94d994f8692b6ff4933f7f542, ab53145cf40647a98bc296fb1fe3ffcd; no voters:
I20260807 00:02:08.168668 6225 raft_consensus.cc:2781] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:08.168787 6225 raft_consensus.cc:482] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:08.168845 6225 raft_consensus.cc:3037] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:08.170006 6225 raft_consensus.cc:504] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
I20260807 00:02:08.170413 6225 leader_election.cc:288] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [CANDIDATE]: Term 1 election: Requested vote from peers ab53145cf40647a98bc296fb1fe3ffcd (127.30.204.66:41175), 88c724a94d994f8692b6ff4933f7f542 (127.30.204.65:45947)
I20260807 00:02:08.170907 5723 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba473a6b8c4f12ab49278e8073d810" candidate_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "88c724a94d994f8692b6ff4933f7f542"
I20260807 00:02:08.170907 5928 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cbba473a6b8c4f12ab49278e8073d810" candidate_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab53145cf40647a98bc296fb1fe3ffcd"
I20260807 00:02:08.171062 5928 raft_consensus.cc:3037] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:08.171062 5723 raft_consensus.cc:3037] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:08.172497 5928 raft_consensus.cc:2454] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a83110be7bd4dd8b1dd9b8516cfdeaa in term 1.
I20260807 00:02:08.172497 5723 raft_consensus.cc:2454] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6a83110be7bd4dd8b1dd9b8516cfdeaa in term 1.
I20260807 00:02:08.172914 6020 leader_election.cc:302] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6a83110be7bd4dd8b1dd9b8516cfdeaa, ab53145cf40647a98bc296fb1fe3ffcd; no voters:
I20260807 00:02:08.173170 6225 raft_consensus.cc:2781] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:08.173427 6225 raft_consensus.cc:686] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 1 LEADER]: Becoming Leader. State: Replica: 6a83110be7bd4dd8b1dd9b8516cfdeaa, State: Running, Role: LEADER
I20260807 00:02:08.173822 6225 consensus_queue.cc:237] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } }
W20260807 00:02:08.174055 6187 tablet.cc:2369] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:02:08.177043 5994 tablet.cc:2369] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:02:08.177417 5517 catalog_manager.cc:5495] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa reported cstate change: term changed from 0 to 1, leader changed from <none> to 6a83110be7bd4dd8b1dd9b8516cfdeaa (127.30.204.67). New cstate: current_term: 1 leader_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "6a83110be7bd4dd8b1dd9b8516cfdeaa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 41999 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 } health_report { overall_health: UNKNOWN } } }
W20260807 00:02:08.188800 5801 tablet.cc:2369] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:02:08.217721 6211 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.213935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34240 (local address 127.30.204.67:41999)
0807 00:02:08.214079 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:08.214084 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:08.214101 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:02:08.214269 (+ 168us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:08.214271 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:08.214328 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:08.214427 (+ 99us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:08.214693 (+ 266us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.215202 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.216136 (+ 934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:08.216355 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:08.216448 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:08.216817 (+ 369us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:08.217279 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:08.217394 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:08.217469 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 00:02:08.217537 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:02:08.224287 5723 raft_consensus.cc:1261] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 1 FOLLOWER]: Refusing update from remote peer 6a83110be7bd4dd8b1dd9b8516cfdeaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:08.224287 5928 raft_consensus.cc:1261] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 1 FOLLOWER]: Refusing update from remote peer 6a83110be7bd4dd8b1dd9b8516cfdeaa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:08.225056 6225 consensus_queue.cc:1035] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab53145cf40647a98bc296fb1fe3ffcd" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41175 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:08.225286 6229 consensus_queue.cc:1035] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [LEADER]: Connected to new peer: Peer: permanent_uuid: "88c724a94d994f8692b6ff4933f7f542" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 45947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:08.234131 6233 mvcc.cc:205] Tried to move back new op lower bound from 7315705561997942784 to 7315705561803051008. Current Snapshot: MvccSnapshot[applied={T|T < 7315705561997942784}]
I20260807 00:02:08.243220 5517 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:08.243450 5517 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:51130:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:08.247035 5517 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7ed1c05775a0439fb5a33ffb4ee27c19: Sending DeleteTablet for 3 replicas of tablet cbba473a6b8c4f12ab49278e8073d810
I20260807 00:02:08.248137 5896 tablet_service.cc:1514] Processing DeleteTablet for tablet cbba473a6b8c4f12ab49278e8073d810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:08 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:59572
I20260807 00:02:08.248142 6082 tablet_service.cc:1514] Processing DeleteTablet for tablet cbba473a6b8c4f12ab49278e8073d810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:08 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:34224
I20260807 00:02:08.248258 5698 tablet_service.cc:1514] Processing DeleteTablet for tablet cbba473a6b8c4f12ab49278e8073d810 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:08 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:38678
I20260807 00:02:08.248769 6249 tablet_replica.cc:330] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: stopping tablet replica
I20260807 00:02:08.248799 6253 tablet_replica.cc:330] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542: stopping tablet replica
I20260807 00:02:08.249159 6249 raft_consensus.cc:2229] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:08.249167 6253 raft_consensus.cc:2229] T cbba473a6b8c4f12ab49278e8073d810 P 88c724a94d994f8692b6ff4933f7f542 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:08.249537 6249 raft_consensus.cc:2258] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:08.249783 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 5606
I20260807 00:02:08.249980 6251 tablet_replica.cc:330] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: stopping tablet replica
I20260807 00:02:08.287492 6249 ts_tablet_manager.cc:1905] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:08.287904 6251 raft_consensus.cc:2229] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:08.288937 6251 raft_consensus.cc:2258] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:08.290884 6251 ts_tablet_manager.cc:1905] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:08.291164 6249 ts_tablet_manager.cc:1918] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:08.291292 6249 log.cc:1196] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/cbba473a6b8c4f12ab49278e8073d810
I20260807 00:02:08.291661 6249 ts_tablet_manager.cc:1939] T cbba473a6b8c4f12ab49278e8073d810 P ab53145cf40647a98bc296fb1fe3ffcd: Deleting consensus metadata
I20260807 00:02:08.293258 5498 catalog_manager.cc:4841] TS ab53145cf40647a98bc296fb1fe3ffcd (127.30.204.66:41175): tablet cbba473a6b8c4f12ab49278e8073d810 (table test-table [id=a3155f564a5f466397554312d5b67d08]) successfully deleted
I20260807 00:02:08.294248 6251 ts_tablet_manager.cc:1918] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:08.294375 6251 log.cc:1196] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/cbba473a6b8c4f12ab49278e8073d810
I20260807 00:02:08.294759 6251 ts_tablet_manager.cc:1939] T cbba473a6b8c4f12ab49278e8073d810 P 6a83110be7bd4dd8b1dd9b8516cfdeaa: Deleting consensus metadata
I20260807 00:02:08.296284 5499 catalog_manager.cc:4841] TS 6a83110be7bd4dd8b1dd9b8516cfdeaa (127.30.204.67:41999): tablet cbba473a6b8c4f12ab49278e8073d810 (table test-table [id=a3155f564a5f466397554312d5b67d08]) successfully deleted
W20260807 00:02:08.296963 5501 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:45947 (error 108)
I20260807 00:02:08.297183 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 5807
I20260807 00:02:08.344133 6202 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.343758 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:45947 (local address 127.0.0.1:38682)
0807 00:02:08.343933 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:08.344030 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:45947: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 00:02:08.344394 5501 catalog_manager.cc:4584] TS 88c724a94d994f8692b6ff4933f7f542 (127.30.204.65:45947): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cbba473a6b8c4f12ab49278e8073d810: Network error: Client connection negotiation failed: client connection to 127.30.204.65:45947: connect: Connection refused (error 111)
I20260807 00:02:08.350848 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 6001
I20260807 00:02:08.363869 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 5474
2026-08-07T00:02:08Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2743 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-08-07T00:02:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:08Z Disabled control of system clock
I20260807 00:02:08.408202 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43127
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:46663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:43127
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 00:02:08.527922 6262 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:08.528245 6262 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:08.528285 6262 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:08.532434 6262 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:08.532510 6262 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:08.532524 6262 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:08.532539 6262 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:08.532553 6262 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:08.537585 6262 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:43127
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43127
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6262
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:08.538734 6262 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:08.540227 6262 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:08.549197 6270 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:08.549201 6268 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:08.549197 6267 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:08.549332 6262 server_base.cc:1034] running on GCE node
I20260807 00:02:08.550200 6262 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:08.551496 6262 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:08.552699 6262 hybrid_clock.cc:648] HybridClock initialized: now 1786060928552680 us; error 37 us; skew 500 ppm
I20260807 00:02:08.554981 6262 webserver.cc:460] Webserver started at http://127.30.204.126:35759/ using document root <none> and password file <none>
I20260807 00:02:08.555708 6262 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:08.555796 6262 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:08.556032 6262 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:08.558420 6262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "e467eaee069c40b69343f021b57e6621"
format_stamp: "Formatted at 2026-08-07 00:02:08 on dist-test-slave-pww0"
server_key: "64c7938de16058f35df5e9751e044310"
server_key_iv: "34ee4efb43bc0be7862d8607199736e0"
server_key_version: "encryptionkey@0"
I20260807 00:02:08.559111 6262 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "e467eaee069c40b69343f021b57e6621"
format_stamp: "Formatted at 2026-08-07 00:02:08 on dist-test-slave-pww0"
server_key: "64c7938de16058f35df5e9751e044310"
server_key_iv: "34ee4efb43bc0be7862d8607199736e0"
server_key_version: "encryptionkey@0"
I20260807 00:02:08.563408 6262 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 00:02:08.566314 6283 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:08.567893 6262 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:08.568064 6262 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "e467eaee069c40b69343f021b57e6621"
format_stamp: "Formatted at 2026-08-07 00:02:08 on dist-test-slave-pww0"
server_key: "64c7938de16058f35df5e9751e044310"
server_key_iv: "34ee4efb43bc0be7862d8607199736e0"
server_key_version: "encryptionkey@0"
I20260807 00:02:08.568202 6262 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:08.573494 6262 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:08.574254 6262 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:08.574440 6262 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:08.581782 6262 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:43127
I20260807 00:02:08.581790 6356 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:43127 every 8 connection(s)
I20260807 00:02:08.582830 6262 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:08.585196 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 6262
I20260807 00:02:08.585326 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:08.585673 31537 external_mini_cluster.cc:1428] Setting key 4eedb9a7cb4a72d977dfc35f342e693a
I20260807 00:02:08.586946 6357 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:08.591984 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.587319 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33964 (local address 127.30.204.126:43127)
0807 00:02:08.588130 (+ 811us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:08.588145 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:08.588187 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:02:08.588889 (+ 702us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:08.588909 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:08.589307 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:08.589682 (+ 375us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:08.590549 (+ 867us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:08.590554 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:08.590574 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:08.590586 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:08.590650 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:08.590669 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:08.590813 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:08.590899 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 00:02:08.591164 (+ 265us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":163,"threads_started":1}
I20260807 00:02:08.593701 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621: Bootstrap starting.
I20260807 00:02:08.596321 6357 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:08.597139 6357 log.cc:824] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:08.599627 6357 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621: No bootstrap required, opened a new log
I20260807 00:02:08.603780 6357 raft_consensus.cc:348] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e467eaee069c40b69343f021b57e6621" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43127 } }
I20260807 00:02:08.604067 6357 raft_consensus.cc:374] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:08.604118 6357 raft_consensus.cc:729] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e467eaee069c40b69343f021b57e6621, State: Initialized, Role: FOLLOWER
I20260807 00:02:08.604560 6357 consensus_queue.cc:260] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e467eaee069c40b69343f021b57e6621" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43127 } }
I20260807 00:02:08.604717 6357 raft_consensus.cc:388] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:08.604768 6357 raft_consensus.cc:482] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:08.604846 6357 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:08.606104 6357 raft_consensus.cc:504] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e467eaee069c40b69343f021b57e6621" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43127 } }
I20260807 00:02:08.606510 6357 leader_election.cc:302] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e467eaee069c40b69343f021b57e6621; no voters:
I20260807 00:02:08.606873 6357 leader_election.cc:288] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:08.607016 6363 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:08.607259 6363 raft_consensus.cc:686] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [term 1 LEADER]: Becoming Leader. State: Replica: e467eaee069c40b69343f021b57e6621, State: Running, Role: LEADER
I20260807 00:02:08.607640 6363 consensus_queue.cc:237] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e467eaee069c40b69343f021b57e6621" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43127 } }
I20260807 00:02:08.608225 6357 sys_catalog.cc:564] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:08.609421 6364 sys_catalog.cc:455] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e467eaee069c40b69343f021b57e6621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e467eaee069c40b69343f021b57e6621" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43127 } } }
I20260807 00:02:08.609409 6365 sys_catalog.cc:455] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e467eaee069c40b69343f021b57e6621. Latest consensus state: current_term: 1 leader_uuid: "e467eaee069c40b69343f021b57e6621" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e467eaee069c40b69343f021b57e6621" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43127 } } }
I20260807 00:02:08.609561 6364 sys_catalog.cc:458] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:08.609578 6365 sys_catalog.cc:458] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:08.609992 6373 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:08.613574 6373 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:08.620290 6373 catalog_manager.cc:1347] Generated new cluster ID: 53efc6964e574276ba049491e5cfe09f
I20260807 00:02:08.620391 6373 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:08.654417 6373 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:08.654855 6373 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:08.661540 6373 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621: Generated new TSK 0
I20260807 00:02:08.662499 6373 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:02:08.675524 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43127
--builtin_ntp_servers=127.30.204.84:46663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260807 00:02:08.798372 6388 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:08.798676 6388 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:08.798717 6388 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:08.802901 6388 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:08.802982 6388 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:08.803041 6388 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:08.808197 6388 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6388
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:08.809516 6388 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:08.811165 6388 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:02:08.821676 6388 server_base.cc:1034] running on GCE node
W20260807 00:02:08.821635 6396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:08.821614 6393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:08.821594 6394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:08.822429 6388 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:08.823138 6388 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:08.824319 6388 hybrid_clock.cc:648] HybridClock initialized: now 1786060928824298 us; error 34 us; skew 500 ppm
I20260807 00:02:08.826292 6388 webserver.cc:460] Webserver started at http://127.30.204.65:37301/ using document root <none> and password file <none>
I20260807 00:02:08.826898 6388 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:08.826967 6388 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:08.827170 6388 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:08.829440 6388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "2406475774b842119bf101ad36565b4b"
format_stamp: "Formatted at 2026-08-07 00:02:08 on dist-test-slave-pww0"
server_key: "bb51b8a01d45bae1c618f7c4dba0c4e6"
server_key_iv: "0ad4e901fa22c9abbfbd8257f3ecff51"
server_key_version: "encryptionkey@0"
I20260807 00:02:08.830070 6388 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "2406475774b842119bf101ad36565b4b"
format_stamp: "Formatted at 2026-08-07 00:02:08 on dist-test-slave-pww0"
server_key: "bb51b8a01d45bae1c618f7c4dba0c4e6"
server_key_iv: "0ad4e901fa22c9abbfbd8257f3ecff51"
server_key_version: "encryptionkey@0"
I20260807 00:02:08.834242 6388 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 00:02:08.837364 6404 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:08.838822 6388 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:08.838961 6388 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "2406475774b842119bf101ad36565b4b"
format_stamp: "Formatted at 2026-08-07 00:02:08 on dist-test-slave-pww0"
server_key: "bb51b8a01d45bae1c618f7c4dba0c4e6"
server_key_iv: "0ad4e901fa22c9abbfbd8257f3ecff51"
server_key_version: "encryptionkey@0"
I20260807 00:02:08.839087 6388 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:08.844152 6388 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:08.844867 6388 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:08.845045 6388 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:08.845635 6388 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:08.846658 6388 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:08.846710 6388 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:08.846765 6388 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:08.846791 6388 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:08.856070 6388 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:43099
I20260807 00:02:08.856092 6561 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:43099 every 8 connection(s)
I20260807 00:02:08.857100 6388 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:08.863745 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 6388
I20260807 00:02:08.863865 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:08.863920 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.859189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:33573 (local address 127.30.204.126:43127)
0807 00:02:08.859338 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:08.859342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:08.860603 (+ 1261us) server_negotiation.cc:407] Connection header received
0807 00:02:08.862114 (+ 1511us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:08.862118 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:08.862171 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:08.862257 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:08.863418 (+ 1161us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:08.863422 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:08.863424 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:08.863439 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:08.863491 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:08.863503 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:08.863595 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:08.863720 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 00:02:08.863772 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:02:08.864200 31537 external_mini_cluster.cc:1428] Setting key 917b928a376f90cbec32ddeef18aeecc
I20260807 00:02:08.865041 6564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:08.859682 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43127 (local address 127.30.204.65:33573)
0807 00:02:08.860434 (+ 752us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:08.860481 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:08.861863 (+ 1382us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:08.862457 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:08.862464 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:08.862563 (+ 99us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:08.863126 (+ 563us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:08.863128 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:08.863171 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:08.863258 (+ 87us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:08.863571 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:08.863580 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:08.863586 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:02:08.863760 (+ 174us) client_negotiation.cc:240] Negotiation successful
0807 00:02:08.864034 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":138,"threads_started":1}
I20260807 00:02:08.866303 6562 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43127
I20260807 00:02:08.866592 6562 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:08.866605 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43127
--builtin_ntp_servers=127.30.204.84:46663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 00:02:08.867136 6562 heartbeater.cc:511] Master 127.30.204.126:43127 requested a full tablet report, sending...
I20260807 00:02:08.869087 6306 ts_manager.cc:194] Registered new tserver with Master: 2406475774b842119bf101ad36565b4b (127.30.204.65:43099)
W20260807 00:02:08.989060 6565 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:08.989403 6565 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:08.989447 6565 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:08.993736 6565 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:08.993816 6565 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:08.993896 6565 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:08.999037 6565 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6565
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:09.000289 6565 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:09.001925 6565 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:09.013005 6573 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:09.012953 6576 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:09.013110 6565 server_base.cc:1034] running on GCE node
W20260807 00:02:09.012951 6578 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:09.013797 6565 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:09.014611 6565 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:09.015784 6565 hybrid_clock.cc:648] HybridClock initialized: now 1786060929015755 us; error 42 us; skew 500 ppm
I20260807 00:02:09.018126 6565 webserver.cc:460] Webserver started at http://127.30.204.66:38121/ using document root <none> and password file <none>
I20260807 00:02:09.018778 6565 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:09.018852 6565 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:09.019063 6565 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:09.021430 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "b60708d82f8a4b328ca0a8276a8b5f60"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "d5c39dc3abde2f4533fdf40a1d172575"
server_key_iv: "1780c5a10ae32f04d1866ab1d36d3281"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.022065 6565 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "b60708d82f8a4b328ca0a8276a8b5f60"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "d5c39dc3abde2f4533fdf40a1d172575"
server_key_iv: "1780c5a10ae32f04d1866ab1d36d3281"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.026474 6565 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:09.029569 6584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:09.031409 6565 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:09.031607 6565 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "b60708d82f8a4b328ca0a8276a8b5f60"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "d5c39dc3abde2f4533fdf40a1d172575"
server_key_iv: "1780c5a10ae32f04d1866ab1d36d3281"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.031749 6565 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:09.037123 6565 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:09.037873 6565 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:09.038064 6565 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:09.038695 6565 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:09.039759 6565 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:09.039813 6565 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:09.039870 6565 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:09.039896 6565 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:09.049340 6565 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:33373
I20260807 00:02:09.049373 6760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:33373 every 8 connection(s)
I20260807 00:02:09.050387 6565 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:09.055239 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 6565
I20260807 00:02:09.055377 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:09.055701 31537 external_mini_cluster.cc:1428] Setting key ffe9b7e981f4056f19d7de20373d0f5f
I20260807 00:02:09.057380 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.052515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:43449 (local address 127.30.204.126:43127)
0807 00:02:09.052665 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.052669 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.053935 (+ 1266us) server_negotiation.cc:407] Connection header received
0807 00:02:09.055379 (+ 1444us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.055382 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.055435 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.055545 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.056902 (+ 1357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.056905 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.056908 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.056921 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.056946 (+ 25us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.056956 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.057063 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.057224 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.057269 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:02:09.057942 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43127
--builtin_ntp_servers=127.30.204.84:46663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260807 00:02:09.058756 6763 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.052962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43127 (local address 127.30.204.66:43449)
0807 00:02:09.053753 (+ 791us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.053799 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.055124 (+ 1325us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.055751 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.055759 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.055883 (+ 124us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:09.056568 (+ 685us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:09.056570 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:09.056619 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:09.056707 (+ 88us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:09.057042 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:09.057053 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:09.057060 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:02:09.057264 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 00:02:09.057596 (+ 332us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":143,"threads_started":1}
I20260807 00:02:09.060210 6761 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43127
I20260807 00:02:09.060464 6761 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:09.060928 6761 heartbeater.cc:511] Master 127.30.204.126:43127 requested a full tablet report, sending...
I20260807 00:02:09.062166 6306 ts_manager.cc:194] Registered new tserver with Master: b60708d82f8a4b328ca0a8276a8b5f60 (127.30.204.66:33373)
W20260807 00:02:09.183069 6765 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:09.183387 6765 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:09.183430 6765 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:09.187736 6765 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:09.187808 6765 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:09.187867 6765 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:09.192987 6765 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43127
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.6765
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:09.194262 6765 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:09.195899 6765 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:09.207624 6778 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:09.207607 6771 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:09.207612 6772 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:09.207803 6765 server_base.cc:1034] running on GCE node
I20260807 00:02:09.208657 6765 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:09.209538 6765 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:09.210750 6765 hybrid_clock.cc:648] HybridClock initialized: now 1786060929210721 us; error 52 us; skew 500 ppm
I20260807 00:02:09.213014 6765 webserver.cc:460] Webserver started at http://127.30.204.67:36109/ using document root <none> and password file <none>
I20260807 00:02:09.213670 6765 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:09.213743 6765 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:09.213959 6765 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:09.216470 6765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "17c765d27cc9fbe83bcd1ba594318108"
server_key_iv: "2b90a7e8f84100b0ef3c8a0e9e41285d"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.217137 6765 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "17c765d27cc9fbe83bcd1ba594318108"
server_key_iv: "2b90a7e8f84100b0ef3c8a0e9e41285d"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.221807 6765 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:09.224877 6785 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:09.226541 6765 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:09.226725 6765 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "17c765d27cc9fbe83bcd1ba594318108"
server_key_iv: "2b90a7e8f84100b0ef3c8a0e9e41285d"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.226874 6765 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:09.232013 6765 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:09.232744 6765 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:09.232930 6765 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:09.233538 6765 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:09.234571 6765 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:09.234622 6765 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:09.234687 6765 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:09.234712 6765 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:09.244299 6765 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:39349
I20260807 00:02:09.244338 6973 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:39349 every 8 connection(s)
I20260807 00:02:09.245335 6765 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:09.246496 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 6765
I20260807 00:02:09.246623 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:09.246944 31537 external_mini_cluster.cc:1428] Setting key 3ded4ff856e3d1c211e7318fbe1bab22
I20260807 00:02:09.252457 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.247859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:44221 (local address 127.30.204.126:43127)
0807 00:02:09.248019 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.248024 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.249248 (+ 1224us) server_negotiation.cc:407] Connection header received
0807 00:02:09.250741 (+ 1493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.250743 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.250788 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.250862 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.252087 (+ 1225us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.252089 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.252092 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.252103 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.252122 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.252129 (+ 7us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.252209 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.252291 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.252333 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:02:09.253455 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.248306 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43127 (local address 127.30.204.67:44221)
0807 00:02:09.249090 (+ 784us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.249137 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.250493 (+ 1356us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.251035 (+ 542us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.251043 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.251164 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:09.251812 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:09.251814 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:09.251862 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:09.251944 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:09.252187 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:09.252191 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:09.252195 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:02:09.252331 (+ 136us) client_negotiation.cc:240] Negotiation successful
0807 00:02:09.252569 (+ 238us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":146,"threads_started":1}
I20260807 00:02:09.254719 6974 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43127
I20260807 00:02:09.254963 6974 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:09.255410 6974 heartbeater.cc:511] Master 127.30.204.126:43127 requested a full tablet report, sending...
I20260807 00:02:09.256805 6306 ts_manager.cc:194] Registered new tserver with Master: fa3f6ec090bb4cc1b8025c14b49baa4a (127.30.204.67:39349)
I20260807 00:02:09.262138 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:02:09.266253 6360 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.264442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33974 (local address 127.30.204.126:43127)
0807 00:02:09.264663 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.264669 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.264796 (+ 127us) server_negotiation.cc:407] Connection header received
0807 00:02:09.264957 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.264961 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.265003 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.265075 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.265852 (+ 777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.265856 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.265859 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.265873 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.265895 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.265904 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.265983 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.266057 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.266104 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":81}
I20260807 00:02:09.269706 6306 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33974:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:09.273002 6306 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:09.285562 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.282837 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45480 (local address 127.30.204.67:39349)
0807 00:02:09.283745 (+ 908us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.283760 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.283785 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:02:09.283839 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.283845 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.284026 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.284191 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.284925 (+ 734us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.284931 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.284943 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.284953 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.284996 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.285024 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.285151 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.285292 (+ 141us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.285379 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":734,"thread_start_us":109,"threads_started":1}
I20260807 00:02:09.285720 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.282265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:39349 (local address 127.0.0.1:45480)
0807 00:02:09.282950 (+ 685us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.282964 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.283050 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.284178 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.284182 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.284198 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:09.284564 (+ 366us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:09.284566 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:09.284589 (+ 23us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:09.284675 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:09.285123 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:09.285133 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:09.285140 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:02:09.285369 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 00:02:09.285479 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":603,"thread_start_us":116,"threads_started":1}
I20260807 00:02:09.285874 7001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.282867 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:43099 (local address 127.0.0.1:34962)
0807 00:02:09.283372 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.283385 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.283474 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.284272 (+ 798us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.284275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.284291 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:09.284708 (+ 417us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:09.284710 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:09.284721 (+ 11us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:09.284749 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:09.285600 (+ 851us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:09.285603 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:09.285607 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:02:09.285679 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:02:09.285718 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":446,"mutex_wait_us":269,"thread_start_us":87,"threads_started":1}
I20260807 00:02:09.286195 6993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.282049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:33373 (local address 127.0.0.1:55922)
0807 00:02:09.282439 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.282490 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.282595 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.283833 (+ 1238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.283840 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.283924 (+ 84us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:09.284813 (+ 889us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:09.284815 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:09.284825 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:09.284849 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:09.285937 (+ 1088us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:09.285940 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:09.285944 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:02:09.286025 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:02:09.286061 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":95,"threads_started":1}
I20260807 00:02:09.286665 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.282212 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55922 (local address 127.30.204.66:33373)
0807 00:02:09.283314 (+ 1102us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.283326 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.283352 (+ 26us) server_negotiation.cc:407] Connection header received
0807 00:02:09.283411 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.283415 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.283649 (+ 234us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.284963 (+ 1314us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.285719 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.285725 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.285736 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.285747 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.285791 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.285820 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.286397 (+ 577us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.286447 (+ 50us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.286519 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":170,"threads_started":1}
I20260807 00:02:09.286701 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.283062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34962 (local address 127.30.204.65:43099)
0807 00:02:09.283893 (+ 831us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.283912 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.283933 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:02:09.283979 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.283983 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.284122 (+ 139us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.284515 (+ 393us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.285405 (+ 890us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.285410 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.285421 (+ 11us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.285430 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.285468 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.285492 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.286151 (+ 659us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.286207 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.286542 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":548,"thread_start_us":162,"threads_started":1}
I20260807 00:02:09.289256 6651 tablet_service.cc:1467] Processing CreateTablet for tablet 61e14c0297f4403aba992d03d8b4bc60 (DEFAULT_TABLE table=test-table [id=af3f0bfa23914515ac493b34ef01e96e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:09.289317 6864 tablet_service.cc:1467] Processing CreateTablet for tablet 61e14c0297f4403aba992d03d8b4bc60 (DEFAULT_TABLE table=test-table [id=af3f0bfa23914515ac493b34ef01e96e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:09.289312 6467 tablet_service.cc:1467] Processing CreateTablet for tablet 61e14c0297f4403aba992d03d8b4bc60 (DEFAULT_TABLE table=test-table [id=af3f0bfa23914515ac493b34ef01e96e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:09.290448 6467 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61e14c0297f4403aba992d03d8b4bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:09.290448 6864 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61e14c0297f4403aba992d03d8b4bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:09.290450 6651 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 61e14c0297f4403aba992d03d8b4bc60. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:09.297046 7002 tablet_bootstrap.cc:492] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Bootstrap starting.
I20260807 00:02:09.298583 7003 tablet_bootstrap.cc:492] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: Bootstrap starting.
I20260807 00:02:09.298610 7005 tablet_bootstrap.cc:492] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Bootstrap starting.
I20260807 00:02:09.299860 7002 tablet_bootstrap.cc:654] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:09.300652 7002 log.cc:824] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:09.300959 7003 tablet_bootstrap.cc:654] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:09.300959 7005 tablet_bootstrap.cc:654] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:09.301864 7005 log.cc:824] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:09.301864 7003 log.cc:824] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:09.302860 7002 tablet_bootstrap.cc:492] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: No bootstrap required, opened a new log
I20260807 00:02:09.303069 7002 ts_tablet_manager.cc:1397] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:02:09.304134 7003 tablet_bootstrap.cc:492] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: No bootstrap required, opened a new log
I20260807 00:02:09.304134 7005 tablet_bootstrap.cc:492] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: No bootstrap required, opened a new log
I20260807 00:02:09.304353 7005 ts_tablet_manager.cc:1397] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:02:09.304353 7003 ts_tablet_manager.cc:1397] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.004s
I20260807 00:02:09.307355 7002 raft_consensus.cc:348] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.307685 7002 raft_consensus.cc:374] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:09.307747 7002 raft_consensus.cc:729] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b60708d82f8a4b328ca0a8276a8b5f60, State: Initialized, Role: FOLLOWER
I20260807 00:02:09.308271 7002 consensus_queue.cc:260] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.309134 6761 heartbeater.cc:503] Master 127.30.204.126:43127 was elected leader, sending a full tablet report...
I20260807 00:02:09.309182 7003 raft_consensus.cc:348] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.309468 7002 ts_tablet_manager.cc:1428] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:02:09.309491 7003 raft_consensus.cc:374] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:09.309545 7003 raft_consensus.cc:729] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2406475774b842119bf101ad36565b4b, State: Initialized, Role: FOLLOWER
I20260807 00:02:09.310053 7003 consensus_queue.cc:260] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.310563 7005 raft_consensus.cc:348] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.310860 7005 raft_consensus.cc:374] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:09.310911 7005 raft_consensus.cc:729] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fa3f6ec090bb4cc1b8025c14b49baa4a, State: Initialized, Role: FOLLOWER
I20260807 00:02:09.311128 6562 heartbeater.cc:503] Master 127.30.204.126:43127 was elected leader, sending a full tablet report...
I20260807 00:02:09.311266 7003 ts_tablet_manager.cc:1428] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 00:02:09.311360 7005 consensus_queue.cc:260] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.312052 6974 heartbeater.cc:503] Master 127.30.204.126:43127 was elected leader, sending a full tablet report...
I20260807 00:02:09.312444 7005 ts_tablet_manager.cc:1428] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Time spent starting tablet: real 0.008s user 0.007s sys 0.000s
W20260807 00:02:09.358366 6563 tablet.cc:2369] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:02:09.369997 7014 raft_consensus.cc:482] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:09.370229 7014 raft_consensus.cc:504] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.371493 7014 leader_election.cc:288] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b60708d82f8a4b328ca0a8276a8b5f60 (127.30.204.66:33373), 2406475774b842119bf101ad36565b4b (127.30.204.65:43099)
I20260807 00:02:09.373660 6977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.371863 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:33373 (local address 127.30.204.67:50541)
0807 00:02:09.371993 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.372011 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.372151 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.372554 (+ 403us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.372558 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.372576 (+ 18us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:09.373053 (+ 477us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:09.373055 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:09.373069 (+ 14us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:09.373103 (+ 34us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:09.373392 (+ 289us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:09.373396 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:09.373399 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:02:09.373482 (+ 83us) client_negotiation.cc:240] Negotiation successful
0807 00:02:09.373520 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260807 00:02:09.373677 6995 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.371979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:50541 (local address 127.30.204.66:33373)
0807 00:02:09.372111 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.372115 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.372135 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:02:09.372315 (+ 180us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.372319 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.372364 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.372445 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.373247 (+ 802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.373251 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.373254 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.373268 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.373294 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.373305 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.373405 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.373492 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.373531 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:02:09.373838 7015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.372182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:43099 (local address 127.30.204.67:55391)
0807 00:02:09.372547 (+ 365us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.372561 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.372654 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.372891 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.372893 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.372908 (+ 15us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:09.373301 (+ 393us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:09.373302 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:09.373312 (+ 10us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:09.373337 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:09.373590 (+ 253us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:09.373593 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:09.373596 (+ 3us) client_negotiation.cc:769] Sending connection context
0807 00:02:09.373668 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:02:09.373703 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":129,"threads_started":1}
I20260807 00:02:09.373872 7000 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.372343 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:55391 (local address 127.30.204.65:43099)
0807 00:02:09.372561 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.372565 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.372633 (+ 68us) server_negotiation.cc:407] Connection header received
0807 00:02:09.372738 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.372741 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.372775 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.372841 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.373459 (+ 618us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.373463 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.373465 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.373479 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.373501 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.373511 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.373593 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.373676 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.373750 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 00:02:09.374493 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e14c0297f4403aba992d03d8b4bc60" candidate_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" is_pre_election: true
I20260807 00:02:09.374538 6495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e14c0297f4403aba992d03d8b4bc60" candidate_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2406475774b842119bf101ad36565b4b" is_pre_election: true
I20260807 00:02:09.374861 6495 raft_consensus.cc:2454] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3f6ec090bb4cc1b8025c14b49baa4a in term 0.
I20260807 00:02:09.374861 6681 raft_consensus.cc:2454] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fa3f6ec090bb4cc1b8025c14b49baa4a in term 0.
I20260807 00:02:09.375357 6790 leader_election.cc:302] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2406475774b842119bf101ad36565b4b, fa3f6ec090bb4cc1b8025c14b49baa4a; no voters:
I20260807 00:02:09.375682 7014 raft_consensus.cc:2781] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:09.375774 7014 raft_consensus.cc:482] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:09.375831 7014 raft_consensus.cc:3037] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:09.376926 7014 raft_consensus.cc:504] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.377312 7014 leader_election.cc:288] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [CANDIDATE]: Term 1 election: Requested vote from peers b60708d82f8a4b328ca0a8276a8b5f60 (127.30.204.66:33373), 2406475774b842119bf101ad36565b4b (127.30.204.65:43099)
I20260807 00:02:09.377827 6681 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e14c0297f4403aba992d03d8b4bc60" candidate_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b60708d82f8a4b328ca0a8276a8b5f60"
I20260807 00:02:09.377869 6495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "61e14c0297f4403aba992d03d8b4bc60" candidate_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2406475774b842119bf101ad36565b4b"
I20260807 00:02:09.377964 6681 raft_consensus.cc:3037] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:09.377997 6495 raft_consensus.cc:3037] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:09.379091 6681 raft_consensus.cc:2454] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3f6ec090bb4cc1b8025c14b49baa4a in term 1.
I20260807 00:02:09.379093 6495 raft_consensus.cc:2454] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fa3f6ec090bb4cc1b8025c14b49baa4a in term 1.
I20260807 00:02:09.379415 6790 leader_election.cc:302] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b60708d82f8a4b328ca0a8276a8b5f60, fa3f6ec090bb4cc1b8025c14b49baa4a; no voters:
I20260807 00:02:09.379650 7014 raft_consensus.cc:2781] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:09.379890 7014 raft_consensus.cc:686] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 1 LEADER]: Becoming Leader. State: Replica: fa3f6ec090bb4cc1b8025c14b49baa4a, State: Running, Role: LEADER
I20260807 00:02:09.380291 7014 consensus_queue.cc:237] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } }
I20260807 00:02:09.383936 6305 catalog_manager.cc:5495] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a reported cstate change: term changed from 0 to 1, leader changed from <none> to fa3f6ec090bb4cc1b8025c14b49baa4a (127.30.204.67). New cstate: current_term: 1 leader_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fa3f6ec090bb4cc1b8025c14b49baa4a" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 39349 } health_report { overall_health: HEALTHY } } }
I20260807 00:02:09.413678 6998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.412195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45486 (local address 127.30.204.67:39349)
0807 00:02:09.412333 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.412336 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.412351 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:09.412460 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.412462 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.412522 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.412600 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.413283 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.413286 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.413289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.413305 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.413328 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.413336 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.413416 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.413469 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.413504 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:02:09.420147 6681 raft_consensus.cc:1261] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 1 FOLLOWER]: Refusing update from remote peer fa3f6ec090bb4cc1b8025c14b49baa4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:09.420147 6495 raft_consensus.cc:1261] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 1 FOLLOWER]: Refusing update from remote peer fa3f6ec090bb4cc1b8025c14b49baa4a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:09.420953 7014 consensus_queue.cc:1035] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "2406475774b842119bf101ad36565b4b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43099 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:09.421200 7016 consensus_queue.cc:1035] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b60708d82f8a4b328ca0a8276a8b5f60" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33373 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:09.430409 7022 mvcc.cc:205] Tried to move back new op lower bound from 7315705566896832512 to 7315705566744674304. Current Snapshot: MvccSnapshot[applied={T|T < 7315705566896832512}]
I20260807 00:02:09.430680 7021 mvcc.cc:205] Tried to move back new op lower bound from 7315705566896832512 to 7315705566744674304. Current Snapshot: MvccSnapshot[applied={T|T < 7315705566896832512}]
I20260807 00:02:09.439644 6305 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:09.439836 6305 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33974:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:09.442454 6305 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e467eaee069c40b69343f021b57e6621: Sending DeleteTablet for 3 replicas of tablet 61e14c0297f4403aba992d03d8b4bc60
I20260807 00:02:09.443544 6651 tablet_service.cc:1514] Processing DeleteTablet for tablet 61e14c0297f4403aba992d03d8b4bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:09 UTC) from {username='slave'} at 127.0.0.1:55922
I20260807 00:02:09.443673 6467 tablet_service.cc:1514] Processing DeleteTablet for tablet 61e14c0297f4403aba992d03d8b4bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:09 UTC) from {username='slave'} at 127.0.0.1:34962
I20260807 00:02:09.444123 6864 tablet_service.cc:1514] Processing DeleteTablet for tablet 61e14c0297f4403aba992d03d8b4bc60 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:09 UTC) from {username='slave'} at 127.0.0.1:45480
I20260807 00:02:09.444243 7034 tablet_replica.cc:330] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b: stopping tablet replica
I20260807 00:02:09.444595 7037 tablet_replica.cc:330] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: stopping tablet replica
I20260807 00:02:09.444633 7034 raft_consensus.cc:2229] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:09.444864 7032 tablet_replica.cc:330] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: stopping tablet replica
I20260807 00:02:09.445060 7034 raft_consensus.cc:2258] T 61e14c0297f4403aba992d03d8b4bc60 P 2406475774b842119bf101ad36565b4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:09.445202 7032 raft_consensus.cc:2229] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:09.445375 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 6388
I20260807 00:02:09.445545 7032 raft_consensus.cc:2258] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:09.471565 7037 raft_consensus.cc:2229] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:09.472584 7037 raft_consensus.cc:2258] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:09.477125 7032 ts_tablet_manager.cc:1905] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:09.477125 7037 ts_tablet_manager.cc:1905] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:09.480225 7032 ts_tablet_manager.cc:1918] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:09.480225 7037 ts_tablet_manager.cc:1918] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:09.480366 7032 log.cc:1196] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/61e14c0297f4403aba992d03d8b4bc60
I20260807 00:02:09.480366 7037 log.cc:1196] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/61e14c0297f4403aba992d03d8b4bc60
I20260807 00:02:09.480723 7037 ts_tablet_manager.cc:1939] T 61e14c0297f4403aba992d03d8b4bc60 P fa3f6ec090bb4cc1b8025c14b49baa4a: Deleting consensus metadata
I20260807 00:02:09.480724 7032 ts_tablet_manager.cc:1939] T 61e14c0297f4403aba992d03d8b4bc60 P b60708d82f8a4b328ca0a8276a8b5f60: Deleting consensus metadata
I20260807 00:02:09.482169 6287 catalog_manager.cc:4841] TS b60708d82f8a4b328ca0a8276a8b5f60 (127.30.204.66:33373): tablet 61e14c0297f4403aba992d03d8b4bc60 (table test-table [id=af3f0bfa23914515ac493b34ef01e96e]) successfully deleted
I20260807 00:02:09.482534 6288 catalog_manager.cc:4841] TS fa3f6ec090bb4cc1b8025c14b49baa4a (127.30.204.67:39349): tablet 61e14c0297f4403aba992d03d8b4bc60 (table test-table [id=af3f0bfa23914515ac493b34ef01e96e]) successfully deleted
W20260807 00:02:09.482952 6287 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:43099 (error 108)
I20260807 00:02:09.483245 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 6565
I20260807 00:02:09.491062 6997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.490694 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:43099 (local address 127.0.0.1:34970)
0807 00:02:09.490855 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.490952 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:43099: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":57}
W20260807 00:02:09.491413 6287 catalog_manager.cc:4584] TS 2406475774b842119bf101ad36565b4b (127.30.204.65:43099): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 61e14c0297f4403aba992d03d8b4bc60: Network error: Client connection negotiation failed: client connection to 127.30.204.65:43099: connect: Connection refused (error 111)
I20260807 00:02:09.495153 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 6765
I20260807 00:02:09.506043 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 6262
2026-08-07T00:02:09Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (1138 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260807 00:02:09.536650 31537 mini_oidc.cc:152] Starting JWKS server
I20260807 00:02:09.538448 31537 webserver.cc:460] Webserver started at https://127.0.0.1:40115/ using document root <none> and password file <none>
I20260807 00:02:09.538641 31537 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 00:02:09.538938 31537 webserver.cc:460] Webserver started at http://127.0.0.1:42813/ using document root <none> and password file <none>
2026-08-07T00:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:09Z Disabled control of system clock
I20260807 00:02:09.549853 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34103
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:43561
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:34103
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40115/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/testchainca.cert with env {}
W20260807 00:02:09.667148 7045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:09.667492 7045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:09.667558 7045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:09.671736 7045 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:09.671818 7045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:09.671838 7045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:09.671860 7045 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:09.671880 7045 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:02:09.671902 7045 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:02:09.671927 7045 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:40115/jwks.json
I20260807 00:02:09.677824 7045 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:43561
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:34103
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34103
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:40115/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:09.679051 7045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:09.680203 7045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:09.689575 7051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:09.689630 7050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:09.689692 7045 server_base.cc:1034] running on GCE node
W20260807 00:02:09.689630 7053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:09.690510 7045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:09.691800 7045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:09.692987 7045 hybrid_clock.cc:648] HybridClock initialized: now 1786060929692962 us; error 40 us; skew 500 ppm
I20260807 00:02:09.695040 7045 webserver.cc:460] Webserver started at http://127.30.204.126:46467/ using document root <none> and password file <none>
I20260807 00:02:09.695717 7045 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:09.695797 7045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:09.696012 7045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:09.698284 7045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "7bf12534c0ad40c28bd33315aa2e0a83"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "5c1ea0810f7d56703ddd20f65bb6e86c"
server_key_iv: "7a5b49cd071be6dc48449d2849b5eaea"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.698881 7045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "7bf12534c0ad40c28bd33315aa2e0a83"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "5c1ea0810f7d56703ddd20f65bb6e86c"
server_key_iv: "7a5b49cd071be6dc48449d2849b5eaea"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.702867 7045 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:09.705631 7071 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:09.707113 7045 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:09.707293 7045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "7bf12534c0ad40c28bd33315aa2e0a83"
format_stamp: "Formatted at 2026-08-07 00:02:09 on dist-test-slave-pww0"
server_key: "5c1ea0810f7d56703ddd20f65bb6e86c"
server_key_iv: "7a5b49cd071be6dc48449d2849b5eaea"
server_key_version: "encryptionkey@0"
I20260807 00:02:09.707417 7045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:09.722983 7045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:09.723969 7045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:09.724195 7045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:09.731678 7045 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:34103
I20260807 00:02:09.731771 7148 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:34103 every 8 connection(s)
I20260807 00:02:09.732753 7045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:09.736414 7149 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:09.736956 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 7045
I20260807 00:02:09.737098 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:09.737377 31537 external_mini_cluster.cc:1428] Setting key 76348aab25577c5a17f70adc719cc246
I20260807 00:02:09.743242 7149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83: Bootstrap starting.
I20260807 00:02:09.746165 7149 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:09.747066 7149 log.cc:824] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:09.748096 7152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.738918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34722 (local address 127.30.204.126:34103)
0807 00:02:09.739721 (+ 803us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.739745 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.739786 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:02:09.740486 (+ 700us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.740505 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.740969 (+ 464us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.741325 (+ 356us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:09.741892 (+ 567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:09.742918 (+ 1026us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:09.744183 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:09.744766 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:09.744945 (+ 179us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:09.745940 (+ 995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:09.745946 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:09.745961 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:09.746497 (+ 536us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:09.746572 (+ 75us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:09.746605 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:09.746702 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:09.746862 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 00:02:09.747168 (+ 306us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":339,"thread_start_us":154,"threads_started":1}
I20260807 00:02:09.749629 7149 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83: No bootstrap required, opened a new log
I20260807 00:02:09.751375 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:02:09.753851 7149 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34103 } }
I20260807 00:02:09.754127 7149 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:09.754199 7149 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7bf12534c0ad40c28bd33315aa2e0a83, State: Initialized, Role: FOLLOWER
I20260807 00:02:09.754642 7149 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34103 } }
I20260807 00:02:09.754812 7149 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:09.754890 7149 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:09.755005 7149 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:09.756155 7149 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34103 } }
I20260807 00:02:09.756534 7149 leader_election.cc:302] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7bf12534c0ad40c28bd33315aa2e0a83; no voters:
I20260807 00:02:09.756901 7149 leader_election.cc:288] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:09.757062 7155 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:09.757294 7155 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [term 1 LEADER]: Becoming Leader. State: Replica: 7bf12534c0ad40c28bd33315aa2e0a83, State: Running, Role: LEADER
I20260807 00:02:09.757647 7155 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34103 } }
I20260807 00:02:09.758222 7149 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:09.759356 7157 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7bf12534c0ad40c28bd33315aa2e0a83. Latest consensus state: current_term: 1 leader_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34103 } } }
I20260807 00:02:09.759548 7157 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:09.759997 7156 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7bf12534c0ad40c28bd33315aa2e0a83" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34103 } } }
I20260807 00:02:09.760169 7156 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:09.760486 7165 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:09.763833 7165 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:09.769802 7165 catalog_manager.cc:1347] Generated new cluster ID: 3171062aed5d48ab9cea3883fa09e4f1
I20260807 00:02:09.769901 7165 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:09.778561 7165 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:09.779947 7165 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:09.788074 7165 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7bf12534c0ad40c28bd33315aa2e0a83: Generated new TSK 0
I20260807 00:02:09.789081 7165 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 00:02:09.821910 7190 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260807 00:02:09.822094 7190 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 00:02:09.822750 7152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:09.803788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34734 (local address 127.30.204.126:34103)
0807 00:02:09.803936 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:09.803941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:09.804166 (+ 225us) server_negotiation.cc:407] Connection header received
0807 00:02:09.804397 (+ 231us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:09.804402 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:09.804472 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:09.804595 (+ 123us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:02:09.804876 (+ 281us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:09.805741 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:09.806662 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:09.807030 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:09.807132 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:09.807528 (+ 396us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:02:09.822259 (+ 14731us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 00:02:09.822573 (+ 314us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:34734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 00:02:09.822827 7152 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:34734: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260807 00:02:09.824126 7189 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:09.803684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34103 (local address 127.0.0.1:34734)
0807 00:02:09.804064 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:09.804078 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:09.804223 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:09.804629 (+ 406us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:09.804631 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:09.804640 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:02:09.804771 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:09.804776 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:09.805886 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:09.805889 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:09.806542 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:09.806551 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:09.807149 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:09.807152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:09.807221 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:09.807279 (+ 58us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:02:09.822569 (+ 15290us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0807 00:02:09.822808 (+ 239us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.204.126:34103: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":130,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.30.204.126:34103: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:40115/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260807 00:02:09.825124 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 7045
2026-08-07T00:02:09Z chronyd exiting
I20260807 00:02:09.875909 31537 test_util.cc:175] -----------------------------------------------
I20260807 00:02:09.876012 31537 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1786060901439409-31537-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (341 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260807 00:02:09.877204 31537 mini_oidc.cc:152] Starting JWKS server
I20260807 00:02:09.878059 31537 webserver.cc:460] Webserver started at https://127.0.0.1:44615/ using document root <none> and password file <none>
I20260807 00:02:09.878146 31537 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 00:02:09.878376 31537 webserver.cc:460] Webserver started at http://127.0.0.1:37869/ using document root <none> and password file <none>
2026-08-07T00:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:09Z Disabled control of system clock
I20260807 00:02:09.889227 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34689
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:45853
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:34689
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44615/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/testchainca.cert with env {}
W20260807 00:02:10.009428 7201 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:10.009747 7201 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:10.009788 7201 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:10.013995 7201 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:10.014081 7201 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:10.014096 7201 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:10.014117 7201 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:10.014133 7201 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:02:10.014154 7201 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:02:10.014166 7201 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:44615/jwks.json
I20260807 00:02:10.020056 7201 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45853
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:34689
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:44615/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7201
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:10.021260 7201 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:10.022330 7201 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:02:10.031699 7201 server_base.cc:1034] running on GCE node
W20260807 00:02:10.031665 7212 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:10.031637 7206 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:10.031641 7207 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:10.032557 7201 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:10.033761 7201 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:10.034963 7201 hybrid_clock.cc:648] HybridClock initialized: now 1786060930034932 us; error 46 us; skew 500 ppm
I20260807 00:02:10.037230 7201 webserver.cc:460] Webserver started at http://127.30.204.126:43845/ using document root <none> and password file <none>
I20260807 00:02:10.037850 7201 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:10.037919 7201 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:10.038132 7201 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:10.040436 7201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "300c746a3bbd497a8b5e1de3ed8a5d45"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "c5031556b568c70576b8bb2fd4fcaa5d"
server_key_iv: "6ef34dbe56d0c7a06fe10958ffe016b7"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.041075 7201 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "300c746a3bbd497a8b5e1de3ed8a5d45"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "c5031556b568c70576b8bb2fd4fcaa5d"
server_key_iv: "6ef34dbe56d0c7a06fe10958ffe016b7"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.045176 7201 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:10.047957 7220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:10.049458 7201 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:10.049618 7201 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "300c746a3bbd497a8b5e1de3ed8a5d45"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "c5031556b568c70576b8bb2fd4fcaa5d"
server_key_iv: "6ef34dbe56d0c7a06fe10958ffe016b7"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.049746 7201 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:10.063982 7201 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:10.064774 7201 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:10.064973 7201 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:10.072672 7201 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:34689
I20260807 00:02:10.072703 7297 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:34689 every 8 connection(s)
I20260807 00:02:10.073762 7201 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:10.076252 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 7201
I20260807 00:02:10.076457 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:10.076821 31537 external_mini_cluster.cc:1428] Setting key ef293f7c9f42ed2f5c929105fed68077
I20260807 00:02:10.077543 7298 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:10.085037 7298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45: Bootstrap starting.
I20260807 00:02:10.086974 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:10.078325 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37856 (local address 127.30.204.126:34689)
0807 00:02:10.079052 (+ 727us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:10.079070 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:10.079119 (+ 49us) server_negotiation.cc:407] Connection header received
0807 00:02:10.079890 (+ 771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:10.079912 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:10.080357 (+ 445us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:10.080677 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:10.081059 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.082070 (+ 1011us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.083339 (+ 1269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.083952 (+ 613us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.084082 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.085033 (+ 951us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:10.085038 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:10.085053 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:10.085523 (+ 470us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:10.085587 (+ 64us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:10.085613 (+ 26us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:10.085699 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:10.085859 (+ 160us) server_negotiation.cc:299] Negotiation successful
0807 00:02:10.086173 (+ 314us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":138,"threads_started":1}
I20260807 00:02:10.087788 7298 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:10.088606 7298 log.cc:824] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:10.090070 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:02:10.091079 7298 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45: No bootstrap required, opened a new log
I20260807 00:02:10.094827 7298 raft_consensus.cc:348] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34689 } }
I20260807 00:02:10.095115 7298 raft_consensus.cc:374] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:10.095199 7298 raft_consensus.cc:729] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 300c746a3bbd497a8b5e1de3ed8a5d45, State: Initialized, Role: FOLLOWER
I20260807 00:02:10.095695 7298 consensus_queue.cc:260] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34689 } }
I20260807 00:02:10.095846 7298 raft_consensus.cc:388] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:10.095916 7298 raft_consensus.cc:482] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:10.096025 7298 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:10.097038 7298 raft_consensus.cc:504] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34689 } }
I20260807 00:02:10.097445 7298 leader_election.cc:302] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 300c746a3bbd497a8b5e1de3ed8a5d45; no voters:
I20260807 00:02:10.097872 7298 leader_election.cc:288] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:10.097990 7308 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:10.098266 7308 raft_consensus.cc:686] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [term 1 LEADER]: Becoming Leader. State: Replica: 300c746a3bbd497a8b5e1de3ed8a5d45, State: Running, Role: LEADER
I20260807 00:02:10.098609 7308 consensus_queue.cc:237] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34689 } }
I20260807 00:02:10.099226 7298 sys_catalog.cc:564] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:10.100301 7311 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 300c746a3bbd497a8b5e1de3ed8a5d45. Latest consensus state: current_term: 1 leader_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34689 } } }
I20260807 00:02:10.100455 7311 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:10.100724 7312 sys_catalog.cc:455] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "300c746a3bbd497a8b5e1de3ed8a5d45" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34689 } } }
I20260807 00:02:10.100814 7312 sys_catalog.cc:458] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:10.100859 7327 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:10.103946 7327 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:10.110136 7327 catalog_manager.cc:1347] Generated new cluster ID: 11b40369cfd84b5e9cd5fd11b0901a6e
I20260807 00:02:10.110227 7327 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:10.131744 7327 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:10.133134 7327 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:10.154104 7327 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 300c746a3bbd497a8b5e1de3ed8a5d45: Generated new TSK 0
I20260807 00:02:10.155200 7327 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260807 00:02:10.181906 7350 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260807 00:02:10.182001 7350 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260807 00:02:10.182829 7304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:10.165678 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37870 (local address 127.30.204.126:34689)
0807 00:02:10.165840 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:10.165845 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:10.165981 (+ 136us) server_negotiation.cc:407] Connection header received
0807 00:02:10.166172 (+ 191us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:10.166175 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:10.166245 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:10.166380 (+ 135us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:02:10.166664 (+ 284us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.167574 (+ 910us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.168514 (+ 940us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.168861 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.168966 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.169312 (+ 346us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:02:10.182347 (+ 13035us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44615/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 00:02:10.182663 (+ 316us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37870: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44615/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":78}
W20260807 00:02:10.182904 7304 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37870: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44615/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260807 00:02:10.182976 7349 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:10.165546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34689 (local address 127.0.0.1:37870)
0807 00:02:10.165887 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:10.165901 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:10.166017 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:10.166417 (+ 400us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:10.166420 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:10.166429 (+ 9us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:02:10.166545 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:10.166550 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.167704 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.167708 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:10.168393 (+ 685us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:10.168403 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.168990 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.168993 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:10.169078 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.169120 (+ 42us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:02:10.182608 (+ 13488us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44615/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0807 00:02:10.182797 (+ 189us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.204.126:34689: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:44615/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":111,"threads_started":1}
I20260807 00:02:10.183826 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 7201
2026-08-07T00:02:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (357 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260807 00:02:10.235189 31537 mini_oidc.cc:152] Starting JWKS server
I20260807 00:02:10.236161 31537 webserver.cc:460] Webserver started at https://127.0.0.1:36171/ using document root <none> and password file <none>
I20260807 00:02:10.236268 31537 mini_oidc.cc:189] Starting OIDC Discovery server
I20260807 00:02:10.236562 31537 webserver.cc:460] Webserver started at http://127.0.0.1:33821/ using document root <none> and password file <none>
2026-08-07T00:02:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:10Z Disabled control of system clock
I20260807 00:02:10.248037 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36019
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:34581
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:36019
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36171/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 00:02:10.366870 7363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:10.367210 7363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:10.367254 7363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:10.371964 7363 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:10.372047 7363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:10.372061 7363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:10.372077 7363 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:10.372092 7363 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:02:10.372107 7363 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:02:10.372130 7363 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:36171/jwks.json
I20260807 00:02:10.377432 7363 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:34581
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:36019
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36019
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:36171/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:10.378629 7363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:10.380133 7363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:10.389428 7371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:10.389480 7374 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:10.389475 7369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:10.389725 7363 server_base.cc:1034] running on GCE node
I20260807 00:02:10.390462 7363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:10.391777 7363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:10.392953 7363 hybrid_clock.cc:648] HybridClock initialized: now 1786060930392933 us; error 37 us; skew 500 ppm
I20260807 00:02:10.395133 7363 webserver.cc:460] Webserver started at http://127.30.204.126:33155/ using document root <none> and password file <none>
I20260807 00:02:10.395824 7363 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:10.395905 7363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:10.396140 7363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:10.398456 7363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "1af03564338341ac8cda4a8668a71dfe"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "154d51b609f1464bba91c319d4447c60"
server_key_iv: "6c2527fc5c6383791933ed9db423083d"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.399094 7363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "1af03564338341ac8cda4a8668a71dfe"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "154d51b609f1464bba91c319d4447c60"
server_key_iv: "6c2527fc5c6383791933ed9db423083d"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.403281 7363 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260807 00:02:10.406142 7386 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:10.407629 7363 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:10.407770 7363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "1af03564338341ac8cda4a8668a71dfe"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "154d51b609f1464bba91c319d4447c60"
server_key_iv: "6c2527fc5c6383791933ed9db423083d"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.407891 7363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:10.423282 7363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:10.424183 7363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:10.424391 7363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:10.432087 7363 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:36019
I20260807 00:02:10.432117 7476 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:36019 every 8 connection(s)
I20260807 00:02:10.433109 7363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:10.435307 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 7363
I20260807 00:02:10.435432 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:10.435761 31537 external_mini_cluster.cc:1428] Setting key 3f677b9c23db6c6190bbe933fe6e564a
I20260807 00:02:10.436376 7477 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:10.444598 7477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe: Bootstrap starting.
I20260807 00:02:10.447417 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:10.437224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37440 (local address 127.30.204.126:36019)
0807 00:02:10.437982 (+ 758us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:10.438004 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:10.438054 (+ 50us) server_negotiation.cc:407] Connection header received
0807 00:02:10.438861 (+ 807us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:10.438888 (+ 27us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:10.439486 (+ 598us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:10.440062 (+ 576us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:10.440609 (+ 547us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.442081 (+ 1472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.443095 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.443881 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.444072 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.445055 (+ 983us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:10.445063 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:10.445082 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:10.445626 (+ 544us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:10.445708 (+ 82us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:10.445741 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:10.445856 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:10.446046 (+ 190us) server_negotiation.cc:299] Negotiation successful
0807 00:02:10.446361 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":306,"thread_start_us":130,"threads_started":1}
I20260807 00:02:10.447817 7477 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:10.448834 7477 log.cc:824] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:10.450889 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:02:10.451411 7477 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe: No bootstrap required, opened a new log
I20260807 00:02:10.455318 7477 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1af03564338341ac8cda4a8668a71dfe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36019 } }
I20260807 00:02:10.455657 7477 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:10.455726 7477 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1af03564338341ac8cda4a8668a71dfe, State: Initialized, Role: FOLLOWER
I20260807 00:02:10.456218 7477 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1af03564338341ac8cda4a8668a71dfe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36019 } }
I20260807 00:02:10.456365 7477 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:10.456436 7477 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:10.456543 7477 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:10.457585 7477 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1af03564338341ac8cda4a8668a71dfe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36019 } }
I20260807 00:02:10.457957 7477 leader_election.cc:302] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1af03564338341ac8cda4a8668a71dfe; no voters:
I20260807 00:02:10.458294 7477 leader_election.cc:288] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:10.458441 7487 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:10.458698 7487 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [term 1 LEADER]: Becoming Leader. State: Replica: 1af03564338341ac8cda4a8668a71dfe, State: Running, Role: LEADER
I20260807 00:02:10.459046 7487 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1af03564338341ac8cda4a8668a71dfe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36019 } }
I20260807 00:02:10.459767 7477 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:10.460989 7489 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1af03564338341ac8cda4a8668a71dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1af03564338341ac8cda4a8668a71dfe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36019 } } }
I20260807 00:02:10.461005 7490 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1af03564338341ac8cda4a8668a71dfe. Latest consensus state: current_term: 1 leader_uuid: "1af03564338341ac8cda4a8668a71dfe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1af03564338341ac8cda4a8668a71dfe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36019 } } }
I20260807 00:02:10.461170 7489 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:10.461171 7490 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:10.461581 7502 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:10.464886 7502 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:10.471540 7502 catalog_manager.cc:1347] Generated new cluster ID: 9b89f7fadc7242d0b1c2b1b0f6c8ecb8
I20260807 00:02:10.471632 7502 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:10.493891 7502 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:10.494921 7502 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:10.500618 7502 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1af03564338341ac8cda4a8668a71dfe: Generated new TSK 0
I20260807 00:02:10.501593 7502 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:02:10.525252 7483 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:10.516754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37442 (local address 127.30.204.126:36019)
0807 00:02:10.516918 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:10.516923 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:10.517031 (+ 108us) server_negotiation.cc:407] Connection header received
0807 00:02:10.517210 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:10.517214 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:10.517279 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:10.517411 (+ 132us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:02:10.517608 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.518455 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.519435 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.519825 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.519933 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.520346 (+ 413us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:02:10.524716 (+ 4370us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36171/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 00:02:10.525067 (+ 351us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:37442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36171/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":81}
W20260807 00:02:10.525343 7483 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:37442: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36171/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260807 00:02:10.526692 7522 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:10.516599 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36019 (local address 127.0.0.1:37442)
0807 00:02:10.516940 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:10.516953 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:10.517065 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:10.517399 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:10.517401 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:10.517407 (+ 6us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:02:10.517511 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:10.517516 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.518617 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.518620 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:10.519307 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:10.519319 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.519983 (+ 664us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.519987 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:10.520070 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.520101 (+ 31us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:02:10.524988 (+ 4887us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36171/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0807 00:02:10.526486 (+ 1498us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.204.126:36019: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:36171/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":1244,"thread_start_us":138,"threads_started":1}
I20260807 00:02:10.527529 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 7363
E20260807 00:02:10.550079 7523 webserver.cc:533] Webserver: sslize failed:
E20260807 00:02:10.550179 7523 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-08-07T00:02:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (346 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-08-07T00:02:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:10Z Disabled control of system clock
I20260807 00:02:10.592226 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39071
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:46157
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:39071
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/testchainca.cert with env {}
W20260807 00:02:10.713292 7528 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:10.713622 7528 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:10.713696 7528 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:10.718081 7528 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:10.718176 7528 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:10.718199 7528 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:10.718220 7528 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:10.718242 7528 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260807 00:02:10.718274 7528 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260807 00:02:10.718302 7528 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260807 00:02:10.724387 7528 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46157
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:39071
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39071
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7528
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:10.725687 7528 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:10.726874 7528 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:10.736487 7537 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:10.736488 7534 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:10.736562 7528 server_base.cc:1034] running on GCE node
W20260807 00:02:10.736487 7533 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:10.737416 7528 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:10.738735 7528 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:10.739986 7528 hybrid_clock.cc:648] HybridClock initialized: now 1786060930739962 us; error 39 us; skew 500 ppm
I20260807 00:02:10.746337 7528 webserver.cc:460] Webserver started at http://127.30.204.126:43671/ using document root <none> and password file <none>
I20260807 00:02:10.747040 7528 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:10.747119 7528 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:10.747349 7528 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:10.749786 7528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "4fc2f93e739d419c974b36c15040a29a"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "37dee9f9243acdb5ca144fad2e0fcffb"
server_key_iv: "3ae29838e86cce30ca126493d4713178"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.750445 7528 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "4fc2f93e739d419c974b36c15040a29a"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "37dee9f9243acdb5ca144fad2e0fcffb"
server_key_iv: "3ae29838e86cce30ca126493d4713178"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.754889 7528 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 00:02:10.757896 7544 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:10.759519 7528 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:10.759697 7528 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "4fc2f93e739d419c974b36c15040a29a"
format_stamp: "Formatted at 2026-08-07 00:02:10 on dist-test-slave-pww0"
server_key: "37dee9f9243acdb5ca144fad2e0fcffb"
server_key_iv: "3ae29838e86cce30ca126493d4713178"
server_key_version: "encryptionkey@0"
I20260807 00:02:10.759832 7528 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:10.776750 7528 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:10.777554 7528 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:10.777765 7528 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:10.785526 7528 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:39071
I20260807 00:02:10.785629 7623 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:39071 every 8 connection(s)
I20260807 00:02:10.786665 7528 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:10.789920 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 7528
I20260807 00:02:10.790017 7624 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:10.790056 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:10.790343 31537 external_mini_cluster.cc:1428] Setting key 1df4c3d30e10e79fe03e65870425e5d1
I20260807 00:02:10.797180 7624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a: Bootstrap starting.
I20260807 00:02:10.799795 7624 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:10.800287 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:10.791838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46772 (local address 127.30.204.126:39071)
0807 00:02:10.792616 (+ 778us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:10.792639 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:10.792692 (+ 53us) server_negotiation.cc:407] Connection header received
0807 00:02:10.793411 (+ 719us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:10.793432 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:10.793886 (+ 454us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:10.794241 (+ 355us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:10.794706 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.795793 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.796744 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.797273 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.797442 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.798316 (+ 874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:10.798321 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:10.798336 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:10.798753 (+ 417us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:10.798820 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:10.798841 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:10.798932 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:10.799093 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:02:10.799384 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":321,"thread_start_us":136,"threads_started":1}
I20260807 00:02:10.800673 7624 log.cc:824] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:10.803059 7624 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a: No bootstrap required, opened a new log
I20260807 00:02:10.803622 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:02:10.807039 7624 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc2f93e739d419c974b36c15040a29a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39071 } }
I20260807 00:02:10.807317 7624 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:10.807381 7624 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4fc2f93e739d419c974b36c15040a29a, State: Initialized, Role: FOLLOWER
I20260807 00:02:10.807920 7624 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc2f93e739d419c974b36c15040a29a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39071 } }
I20260807 00:02:10.808079 7624 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:10.808158 7624 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:10.808262 7624 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:10.809267 7624 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc2f93e739d419c974b36c15040a29a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39071 } }
I20260807 00:02:10.809631 7624 leader_election.cc:302] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4fc2f93e739d419c974b36c15040a29a; no voters:
I20260807 00:02:10.809975 7624 leader_election.cc:288] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:10.810104 7632 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:10.810364 7632 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [term 1 LEADER]: Becoming Leader. State: Replica: 4fc2f93e739d419c974b36c15040a29a, State: Running, Role: LEADER
I20260807 00:02:10.810703 7632 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc2f93e739d419c974b36c15040a29a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39071 } }
I20260807 00:02:10.811287 7624 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:10.812453 7633 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4fc2f93e739d419c974b36c15040a29a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc2f93e739d419c974b36c15040a29a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39071 } } }
I20260807 00:02:10.812616 7633 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:10.812914 7636 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4fc2f93e739d419c974b36c15040a29a. Latest consensus state: current_term: 1 leader_uuid: "4fc2f93e739d419c974b36c15040a29a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4fc2f93e739d419c974b36c15040a29a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39071 } } }
I20260807 00:02:10.813028 7636 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:10.813102 7645 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:10.816624 7645 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:10.823496 7645 catalog_manager.cc:1347] Generated new cluster ID: 2a9d04d1604140e4b5b211dbf0d8108c
I20260807 00:02:10.823589 7645 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:10.837513 7645 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:10.838555 7645 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:10.847364 7645 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4fc2f93e739d419c974b36c15040a29a: Generated new TSK 0
I20260807 00:02:10.848345 7645 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:02:10.920626 7630 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:10.866038 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46788 (local address 127.30.204.126:39071)
0807 00:02:10.866193 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:10.866199 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:10.866409 (+ 210us) server_negotiation.cc:407] Connection header received
0807 00:02:10.866638 (+ 229us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:10.866642 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:10.866730 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:10.866833 (+ 103us) server_negotiation.cc:226] Negotiated authn=JWT
0807 00:02:10.867089 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.867719 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.868949 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.869179 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.869256 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.869600 (+ 344us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0807 00:02:10.920197 (+ 50597us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 00:02:10.920494 (+ 297us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:46788: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 00:02:10.920717 7630 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:46788: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260807 00:02:10.920938 7676 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:10.865949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39071 (local address 127.0.0.1:46788)
0807 00:02:10.866304 (+ 355us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:10.866319 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:10.866460 (+ 141us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:10.866848 (+ 388us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:10.866851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:10.866859 (+ 8us) client_negotiation.cc:189] Negotiated authn=JWT
0807 00:02:10.866985 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:10.866991 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.867827 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.867832 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:10.868837 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:10.868849 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:10.869298 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:10.869303 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:10.869402 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:10.869436 (+ 34us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0807 00:02:10.920487 (+ 51051us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0807 00:02:10.920733 (+ 246us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.30.204.126:39071: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":147,"threads_started":1}
I20260807 00:02:10.921669 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 7528
2026-08-07T00:02:10Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (364 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (147 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (135 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:11 dist-test-slave-pww0 krb5kdc[7693](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:11 dist-test-slave-pww0 krb5kdc[7693](info): set up 2 sockets
Aug 07 00:02:11 dist-test-slave-pww0 krb5kdc[7693](info): commencing operation
krb5kdc: starting...
W20260807 00:02:12.368296 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.008s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:12 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:12.506734 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:33859
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:33859
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:12.627817 7714 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:12.628118 7714 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:12.628180 7714 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:12.632292 7714 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:12.632366 7714 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:12.632393 7714 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:12.632413 7714 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:12.632434 7714 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:12.637984 7714 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:33859
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:33859
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7714
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:12.639206 7714 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:12.640417 7714 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:12.649673 7721 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:12.649675 7724 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:12.649729 7722 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:12.649819 7714 server_base.cc:1034] running on GCE node
I20260807 00:02:12.650703 7714 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:12.652020 7714 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:12.653226 7714 hybrid_clock.cc:648] HybridClock initialized: now 1786060932653206 us; error 39 us; skew 500 ppm
Aug 07 00:02:12 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:12.656579 7714 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:12.657790 7714 webserver.cc:460] Webserver started at http://127.30.204.126:44241/ using document root <none> and password file <none>
I20260807 00:02:12.658481 7714 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:12.658597 7714 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:12.658838 7714 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:12.661209 7714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "fac5474c4e79413f8b566379825187c5"
format_stamp: "Formatted at 2026-08-07 00:02:12 on dist-test-slave-pww0"
server_key: "55ed03f03aaab1069cef6bae0f213b3f"
server_key_iv: "5b5e17d2dd3b7c7437e6761d7016c9a2"
server_key_version: "encryptionkey@0"
I20260807 00:02:12.661824 7714 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "fac5474c4e79413f8b566379825187c5"
format_stamp: "Formatted at 2026-08-07 00:02:12 on dist-test-slave-pww0"
server_key: "55ed03f03aaab1069cef6bae0f213b3f"
server_key_iv: "5b5e17d2dd3b7c7437e6761d7016c9a2"
server_key_version: "encryptionkey@0"
I20260807 00:02:12.666172 7714 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 00:02:12.668995 7735 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:12.670423 7714 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:12.670583 7714 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "fac5474c4e79413f8b566379825187c5"
format_stamp: "Formatted at 2026-08-07 00:02:12 on dist-test-slave-pww0"
server_key: "55ed03f03aaab1069cef6bae0f213b3f"
server_key_iv: "5b5e17d2dd3b7c7437e6761d7016c9a2"
server_key_version: "encryptionkey@0"
I20260807 00:02:12.670712 7714 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:12.701707 7714 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:12.705523 7714 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:12.705770 7714 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:12.713402 7714 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:33859
I20260807 00:02:12.713411 7818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:33859 every 8 connection(s)
I20260807 00:02:12.714463 7714 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:12.715395 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 7714
I20260807 00:02:12.715556 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:12.715883 31537 external_mini_cluster.cc:1428] Setting key 7fc729da10809b2cb6c54184250b1115
I20260807 00:02:12.718166 7820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:12.725786 7820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5: Bootstrap starting.
Aug 07 00:02:12 dist-test-slave-pww0 krb5kdc[7693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060932, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:12.728519 7820 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:12.729336 7820 log.cc:824] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:12.732092 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:12.717396 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53986 (local address 127.30.204.126:33859)
0807 00:02:12.717864 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:12.717881 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:12.717920 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:02:12.718626 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:12.718646 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:12.719002 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:12.719318 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:12.720327 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:12.721471 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:12.722672 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:12.723199 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:12.723335 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:12.726691 (+ 3356us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:12.726698 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:12.726714 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:12.726753 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:12.728572 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:12.729086 (+ 514us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:12.729093 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:12.729098 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:12.729235 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:12.729559 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:12.729563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:12.729565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:12.730095 (+ 530us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:12.730277 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:12.730684 (+ 407us) server_negotiation.cc:299] Negotiation successful
0807 00:02:12.731024 (+ 340us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":318,"thread_start_us":146,"threads_started":1}
I20260807 00:02:12.732197 7820 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5: No bootstrap required, opened a new log
I20260807 00:02:12.736336 7820 raft_consensus.cc:348] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac5474c4e79413f8b566379825187c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33859 } }
I20260807 00:02:12.736601 7820 raft_consensus.cc:374] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:12.736663 7820 raft_consensus.cc:729] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fac5474c4e79413f8b566379825187c5, State: Initialized, Role: FOLLOWER
I20260807 00:02:12.737074 7820 consensus_queue.cc:260] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac5474c4e79413f8b566379825187c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33859 } }
I20260807 00:02:12.737195 7820 raft_consensus.cc:388] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:12.737246 7820 raft_consensus.cc:482] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:12.737320 7820 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:12.738304 7820 raft_consensus.cc:504] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac5474c4e79413f8b566379825187c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33859 } }
I20260807 00:02:12.738649 7820 leader_election.cc:302] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fac5474c4e79413f8b566379825187c5; no voters:
I20260807 00:02:12.738947 7820 leader_election.cc:288] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:12.739136 7826 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:12.739382 7826 raft_consensus.cc:686] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [term 1 LEADER]: Becoming Leader. State: Replica: fac5474c4e79413f8b566379825187c5, State: Running, Role: LEADER
I20260807 00:02:12.739727 7826 consensus_queue.cc:237] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac5474c4e79413f8b566379825187c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33859 } }
I20260807 00:02:12.740299 7820 sys_catalog.cc:564] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:12.741539 7830 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader fac5474c4e79413f8b566379825187c5. Latest consensus state: current_term: 1 leader_uuid: "fac5474c4e79413f8b566379825187c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac5474c4e79413f8b566379825187c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33859 } } }
I20260807 00:02:12.741665 7830 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:12.741513 7828 sys_catalog.cc:455] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fac5474c4e79413f8b566379825187c5" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fac5474c4e79413f8b566379825187c5" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33859 } } }
I20260807 00:02:12.741844 7828 sys_catalog.cc:458] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:12.742204 7842 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:12.745972 7842 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:12.752883 7842 catalog_manager.cc:1347] Generated new cluster ID: a901455355194925af2b5121d8ddeeeb
I20260807 00:02:12.752982 7842 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:12.760281 7842 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:12.761476 7842 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:12.769498 7842 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fac5474c4e79413f8b566379825187c5: Generated new TSK 0
I20260807 00:02:12.770447 7842 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:02:12.814733 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:33859
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:12.943025 7860 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:12.943325 7860 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:12.943389 7860 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:12.947650 7860 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:12.947728 7860 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:12.947813 7860 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:12.953593 7860 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:33859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.7860
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:12.954828 7860 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:12.956012 7860 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:12.966878 7872 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:12.966938 7860 server_base.cc:1034] running on GCE node
W20260807 00:02:12.966897 7873 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:12.966878 7875 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:12.967689 7860 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:12.968407 7860 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:12.969596 7860 hybrid_clock.cc:648] HybridClock initialized: now 1786060932969552 us; error 57 us; skew 500 ppm
Aug 07 00:02:12 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:12.972749 7860 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:02:12.973793 7860 webserver.cc:460] Webserver started at http://127.30.204.65:32935/ using document root <none> and password file <none>
I20260807 00:02:12.974393 7860 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:12.974459 7860 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:12.974671 7860 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:12.977015 7860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "808050afeada4b21afff07e14ebd82b9"
format_stamp: "Formatted at 2026-08-07 00:02:12 on dist-test-slave-pww0"
server_key: "3da07a82587df9ffcad3ee9964bccbc4"
server_key_iv: "93fcfe76315e513b0dd9ace5be8340ab"
server_key_version: "encryptionkey@0"
I20260807 00:02:12.977625 7860 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "808050afeada4b21afff07e14ebd82b9"
format_stamp: "Formatted at 2026-08-07 00:02:12 on dist-test-slave-pww0"
server_key: "3da07a82587df9ffcad3ee9964bccbc4"
server_key_iv: "93fcfe76315e513b0dd9ace5be8340ab"
server_key_version: "encryptionkey@0"
I20260807 00:02:12.981846 7860 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.001s
I20260807 00:02:12.984694 7883 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:12.986018 7860 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:12.986150 7860 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "808050afeada4b21afff07e14ebd82b9"
format_stamp: "Formatted at 2026-08-07 00:02:12 on dist-test-slave-pww0"
server_key: "3da07a82587df9ffcad3ee9964bccbc4"
server_key_iv: "93fcfe76315e513b0dd9ace5be8340ab"
server_key_version: "encryptionkey@0"
I20260807 00:02:12.986269 7860 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:13.003288 7860 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:13.006743 7860 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:13.006947 7860 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:13.007632 7860 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:13.008572 7860 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:13.008620 7860 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.008663 7860 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:13.008692 7860 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.018050 7860 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:33041
I20260807 00:02:13.018075 8060 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:33041 every 8 connection(s)
I20260807 00:02:13.019021 7860 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:13.023977 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 7860
I20260807 00:02:13.024121 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:13.024410 31537 external_mini_cluster.cc:1428] Setting key 178a50a87257d3d5e0f9c4b34e96e1ee
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060932, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:13.035136 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.021019 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:51997 (local address 127.30.204.126:33859)
0807 00:02:13.021160 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.021164 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.022087 (+ 923us) server_negotiation.cc:407] Connection header received
0807 00:02:13.023603 (+ 1516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.023606 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.023670 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.023782 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.025238 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.025803 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.027536 (+ 1733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.027790 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.027876 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.030734 (+ 2858us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:13.030737 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:13.030739 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:13.030787 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:13.032482 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.033190 (+ 708us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.033193 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.033194 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.033282 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.033748 (+ 466us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.033751 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.033752 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.033996 (+ 244us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:13.034085 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:13.034857 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 00:02:13.034996 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:13.036329 8064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.021435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:33859 (local address 127.30.204.65:51997)
0807 00:02:13.021919 (+ 484us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:13.021966 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:13.023346 (+ 1380us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:13.023982 (+ 636us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:13.023991 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:13.024429 (+ 438us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:13.025086 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:13.025106 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.025936 (+ 830us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.025940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:13.027343 (+ 1403us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:13.027358 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.027958 (+ 600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.027961 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:13.028183 (+ 222us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.028831 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:13.028850 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:13.030519 (+ 1669us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:13.032635 (+ 2116us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:13.032644 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:13.032665 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:13.033086 (+ 421us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:13.033373 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:13.033376 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:13.033378 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:13.033608 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:13.034114 (+ 506us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:13.034122 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:13.034520 (+ 398us) client_negotiation.cc:769] Sending connection context
0807 00:02:13.034784 (+ 264us) client_negotiation.cc:240] Negotiation successful
0807 00:02:13.035209 (+ 425us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":117,"threads_started":1}
I20260807 00:02:13.037766 8062 heartbeater.cc:347] Connected to a master server at 127.30.204.126:33859
I20260807 00:02:13.038043 8062 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:02:13.038743 8062 heartbeater.cc:511] Master 127.30.204.126:33859 requested a full tablet report, sending...
I20260807 00:02:13.040659 7762 ts_manager.cc:194] Registered new tserver with Master: 808050afeada4b21afff07e14ebd82b9 (127.30.204.65:33041)
I20260807 00:02:13.042325 7762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:51997
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:02:13.059281 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:33859
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:13.181706 8069 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:13.182039 8069 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:13.182106 8069 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:13.186492 8069 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:13.186580 8069 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:13.186669 8069 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:13.192382 8069 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:33859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8069
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:13.193607 8069 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:13.194744 8069 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:13.205269 8075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:13.205346 8069 server_base.cc:1034] running on GCE node
W20260807 00:02:13.205269 8077 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:13.205289 8079 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:13.205992 8069 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:13.206713 8069 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:13.207908 8069 hybrid_clock.cc:648] HybridClock initialized: now 1786060933207876 us; error 46 us; skew 500 ppm
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.210990 8069 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:02:13.212103 8069 webserver.cc:460] Webserver started at http://127.30.204.66:43457/ using document root <none> and password file <none>
I20260807 00:02:13.212703 8069 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:13.212772 8069 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:13.212996 8069 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:13.215338 8069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "a68468d8b12448f3bb95d3c7a839d7c3"
format_stamp: "Formatted at 2026-08-07 00:02:13 on dist-test-slave-pww0"
server_key: "e2b28ff12c808ddcbe2da4d8eda8d1ea"
server_key_iv: "8de5919d686350f5fcc7cfc1503e9c4d"
server_key_version: "encryptionkey@0"
I20260807 00:02:13.216042 8069 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "a68468d8b12448f3bb95d3c7a839d7c3"
format_stamp: "Formatted at 2026-08-07 00:02:13 on dist-test-slave-pww0"
server_key: "e2b28ff12c808ddcbe2da4d8eda8d1ea"
server_key_iv: "8de5919d686350f5fcc7cfc1503e9c4d"
server_key_version: "encryptionkey@0"
I20260807 00:02:13.220495 8069 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:02:13.223414 8086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.225016 8069 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:13.225169 8069 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "a68468d8b12448f3bb95d3c7a839d7c3"
format_stamp: "Formatted at 2026-08-07 00:02:13 on dist-test-slave-pww0"
server_key: "e2b28ff12c808ddcbe2da4d8eda8d1ea"
server_key_iv: "8de5919d686350f5fcc7cfc1503e9c4d"
server_key_version: "encryptionkey@0"
I20260807 00:02:13.225310 8069 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:13.239127 8069 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:13.243175 8069 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:13.243403 8069 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:13.244138 8069 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:13.245142 8069 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:13.245198 8069 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.245249 8069 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:13.245283 8069 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.255007 8069 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:44873
I20260807 00:02:13.255095 8258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:44873 every 8 connection(s)
I20260807 00:02:13.256227 8069 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:13.257372 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 8069
I20260807 00:02:13.257479 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:13.257745 31537 external_mini_cluster.cc:1428] Setting key c898a5db06aaa7f694078ef2c782fbc0
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:02:13.272181 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.258716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:46631 (local address 127.30.204.126:33859)
0807 00:02:13.258891 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.258896 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.259813 (+ 917us) server_negotiation.cc:407] Connection header received
0807 00:02:13.261282 (+ 1469us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.261286 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.261348 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.261427 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.262741 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.263289 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.264551 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.264788 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.264885 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.267816 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:13.267819 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:13.267822 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:13.267853 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:13.269465 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.270261 (+ 796us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.270273 (+ 12us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.270275 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.270371 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.270851 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.270855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.270857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.271070 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:13.271154 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:13.271923 (+ 769us) server_negotiation.cc:299] Negotiation successful
0807 00:02:13.272046 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 00:02:13.273367 8261 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.259018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:33859 (local address 127.30.204.66:46631)
0807 00:02:13.259659 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:13.259712 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:13.261042 (+ 1330us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:13.261634 (+ 592us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:13.261649 (+ 15us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:13.262045 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:13.262573 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:13.262588 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.263427 (+ 839us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.263430 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:13.264415 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:13.264427 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.264951 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.264955 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:13.265140 (+ 185us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.265840 (+ 700us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:13.265859 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:13.267591 (+ 1732us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:13.269676 (+ 2085us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:13.269682 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:13.269698 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:13.270132 (+ 434us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:13.270488 (+ 356us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:13.270491 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:13.270493 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:13.270729 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:13.271211 (+ 482us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:13.271219 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:13.271633 (+ 414us) client_negotiation.cc:769] Sending connection context
0807 00:02:13.271859 (+ 226us) client_negotiation.cc:240] Negotiation successful
0807 00:02:13.272261 (+ 402us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":142,"threads_started":1}
I20260807 00:02:13.274708 8259 heartbeater.cc:347] Connected to a master server at 127.30.204.126:33859
I20260807 00:02:13.274999 8259 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:13.275741 8259 heartbeater.cc:511] Master 127.30.204.126:33859 requested a full tablet report, sending...
I20260807 00:02:13.277181 7762 ts_manager.cc:194] Registered new tserver with Master: a68468d8b12448f3bb95d3c7a839d7c3 (127.30.204.66:44873)
I20260807 00:02:13.278016 7762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:46631
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:02:13.291399 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:33859
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36025
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:13.413347 8266 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:13.413676 8266 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:13.413741 8266 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:13.417979 8266 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:13.418066 8266 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:13.418156 8266 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:13.423861 8266 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36025
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:33859
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8266
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:13.425117 8266 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:13.426266 8266 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:13.436602 8271 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:13.436698 8275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:13.436719 8273 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:13.437218 8266 server_base.cc:1034] running on GCE node
I20260807 00:02:13.437711 8266 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:13.438436 8266 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:13.439599 8266 hybrid_clock.cc:648] HybridClock initialized: now 1786060933439560 us; error 47 us; skew 500 ppm
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.442579 8266 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:02:13.443661 8266 webserver.cc:460] Webserver started at http://127.30.204.67:39809/ using document root <none> and password file <none>
I20260807 00:02:13.444248 8266 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:13.444322 8266 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:13.444541 8266 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:13.446914 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "340541ea1c574f2ea13c75fcce888287"
format_stamp: "Formatted at 2026-08-07 00:02:13 on dist-test-slave-pww0"
server_key: "80eee069033fc3295c17c6af808e7912"
server_key_iv: "e7439f54e98444f7dfe15d0e7b9e3a63"
server_key_version: "encryptionkey@0"
I20260807 00:02:13.447587 8266 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "340541ea1c574f2ea13c75fcce888287"
format_stamp: "Formatted at 2026-08-07 00:02:13 on dist-test-slave-pww0"
server_key: "80eee069033fc3295c17c6af808e7912"
server_key_iv: "e7439f54e98444f7dfe15d0e7b9e3a63"
server_key_version: "encryptionkey@0"
I20260807 00:02:13.451897 8266 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:13.454742 8285 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.456246 8266 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:13.456387 8266 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "340541ea1c574f2ea13c75fcce888287"
format_stamp: "Formatted at 2026-08-07 00:02:13 on dist-test-slave-pww0"
server_key: "80eee069033fc3295c17c6af808e7912"
server_key_iv: "e7439f54e98444f7dfe15d0e7b9e3a63"
server_key_version: "encryptionkey@0"
I20260807 00:02:13.456506 8266 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:13.472625 8266 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:13.476037 8266 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:13.476241 8266 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:13.476899 8266 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:13.477790 8266 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:13.477842 8266 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.477900 8266 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:13.477928 8266 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:13.487159 8266 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:44307
I20260807 00:02:13.487192 8453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:44307 every 8 connection(s)
I20260807 00:02:13.488196 8266 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:13.489838 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 8266
I20260807 00:02:13.489964 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:13.490340 31537 external_mini_cluster.cc:1428] Setting key aac4ca432915e903763dec85aaa45338
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:13.503196 7824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.490279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:40121 (local address 127.30.204.126:33859)
0807 00:02:13.490444 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.490449 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.491589 (+ 1140us) server_negotiation.cc:407] Connection header received
0807 00:02:13.493135 (+ 1546us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.493138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.493189 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.493261 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.494494 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.495065 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.496406 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.496634 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.496722 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.499280 (+ 2558us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:13.499283 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:13.499285 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:13.499314 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:13.500832 (+ 1518us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.501454 (+ 622us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.501458 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.501459 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.501557 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.501956 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.501959 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.501960 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.502215 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:13.502322 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:13.502914 (+ 592us) server_negotiation.cc:299] Negotiation successful
0807 00:02:13.503063 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:02:13.504309 8461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.490812 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:33859 (local address 127.30.204.67:40121)
0807 00:02:13.491404 (+ 592us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:13.491488 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:13.492915 (+ 1427us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:13.493423 (+ 508us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:13.493430 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:13.493825 (+ 395us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:13.494347 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:13.494362 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.495189 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.495191 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:13.496262 (+ 1071us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:13.496274 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.496799 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.496801 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:13.496965 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.497572 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:13.497591 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:13.499085 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:13.501013 (+ 1928us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:13.501018 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:13.501033 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:13.501332 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:13.501673 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:13.501676 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:13.501677 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:13.501834 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:13.502321 (+ 487us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:13.502327 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:13.502600 (+ 273us) client_negotiation.cc:769] Sending connection context
0807 00:02:13.502797 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 00:02:13.503138 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":165,"threads_started":1}
I20260807 00:02:13.505673 8456 heartbeater.cc:347] Connected to a master server at 127.30.204.126:33859
I20260807 00:02:13.505949 8456 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:13.506510 8456 heartbeater.cc:511] Master 127.30.204.126:33859 requested a full tablet report, sending...
I20260807 00:02:13.507725 7762 ts_manager.cc:194] Registered new tserver with Master: 340541ea1c574f2ea13c75fcce888287 (127.30.204.67:44307)
I20260807 00:02:13.508292 7762 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:40121
I20260807 00:02:13.516902 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/test-admin.keytab.
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.536037 31537 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260807 00:02:13.539039 8466 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:13.537606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56464 (local address 127.30.204.65:33041)
0807 00:02:13.537905 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.537913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.537943 (+ 30us) server_negotiation.cc:407] Connection header received
0807 00:02:13.538021 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.538027 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.538224 (+ 197us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.538369 (+ 145us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.538873 (+ 504us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56464: BlockingRecv error: recv got EOF from 127.0.0.1:56464 (error 108)
Metrics: {"server-negotiator.queue_time_us":184,"thread_start_us":88,"threads_started":1}
W20260807 00:02:13.539376 31537 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc)
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.540534 31537 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.541658 31537 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:02:13.552392 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.542726 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56476 (local address 127.30.204.65:33041)
0807 00:02:13.542882 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.542886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.542951 (+ 65us) server_negotiation.cc:407] Connection header received
0807 00:02:13.543066 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.543068 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.543130 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.543200 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.543934 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.544732 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.545640 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.546100 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.546211 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.548801 (+ 2590us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:13.548816 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:13.548833 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:13.548867 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:13.550418 (+ 1551us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.550954 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.550960 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.550962 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.551060 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.551284 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.551286 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.551287 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.551605 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:13.551801 (+ 196us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:13.552029 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 00:02:13.552208 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
W20260807 00:02:13.556281 8466 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:13.555367 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56478 (local address 127.30.204.65:33041)
0807 00:02:13.555540 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.555544 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.555611 (+ 67us) server_negotiation.cc:407] Connection header received
0807 00:02:13.555717 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.555719 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.555760 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.555825 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.556165 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56478: BlockingRecv error: recv got EOF from 127.0.0.1:56478 (error 108)
Metrics: {"server-negotiator.queue_time_us":61}
W20260807 00:02:13.556397 31537 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc' of type FILE
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.557439 31537 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.558292 31537 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:02:13.567668 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.559208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56492 (local address 127.30.204.65:33041)
0807 00:02:13.559358 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.559361 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.559376 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:13.559585 (+ 209us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.559590 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.559654 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.559733 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.560290 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.560873 (+ 583us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.561724 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.561992 (+ 268us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.562105 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.564416 (+ 2311us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:13.564419 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:13.564421 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:13.564449 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:13.565780 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.566209 (+ 429us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.566212 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.566213 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.566301 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.566650 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.566653 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.566654 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.566844 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:13.566967 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:13.567279 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 00:02:13.567407 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
W20260807 00:02:13.570603 8466 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:13.569568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56498 (local address 127.30.204.65:33041)
0807 00:02:13.569718 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.569722 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.569794 (+ 72us) server_negotiation.cc:407] Connection header received
0807 00:02:13.569919 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.569922 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.569966 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.570031 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.570486 (+ 455us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:56498: BlockingRecv error: recv got EOF from 127.0.0.1:56498 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260807 00:02:13.570710 31537 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1786060901439409-31537-0/krb5kdc/krb5cc' of type FILE
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.571794 31537 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:13.572701 31537 init.cc:295] Successfully reacquired a new kerberos TGT
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[7693](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060933, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:02:13.582063 8466 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:13.573789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56514 (local address 127.30.204.65:33041)
0807 00:02:13.573935 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:13.573941 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:13.573993 (+ 52us) server_negotiation.cc:407] Connection header received
0807 00:02:13.574144 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:13.574148 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:13.574195 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:13.574263 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:13.574811 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.575338 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.576275 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:13.576504 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:13.576597 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:13.579074 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:13.579078 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:13.579081 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:13.579113 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:13.580403 (+ 1290us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.580844 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.580848 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.580849 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.580932 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:13.581206 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:13.581210 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:13.581212 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:13.581403 (+ 191us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:13.581550 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:13.581783 (+ 233us) server_negotiation.cc:299] Negotiation successful
0807 00:02:13.581900 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 00:02:13.583263 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 7860
I20260807 00:02:13.606608 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 8069
I20260807 00:02:13.623600 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 8266
I20260807 00:02:13.634330 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 7714
2026-08-07T00:02:13Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2440 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[8487](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[8487](info): set up 2 sockets
Aug 07 00:02:13 dist-test-slave-pww0 krb5kdc[8487](info): commencing operation
krb5kdc: starting...
W20260807 00:02:14.808406 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.001s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:14 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:14Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:14Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab.
Entry for principal oryx/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab.
I20260807 00:02:14.948263 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:35533
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab
--principal=oryx/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:35533
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:15.073182 8502 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:15.073503 8502 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:15.073565 8502 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:15.077911 8502 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:15.077994 8502 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:15.078023 8502 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:15.078044 8502 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:15.078066 8502 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:15.083926 8502 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:35533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab
--principal=oryx/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:35533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8502
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:15.085144 8502 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:15.086333 8502 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:02:15.095633 8502 server_base.cc:1034] running on GCE node
W20260807 00:02:15.095567 8511 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:15.095567 8508 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:15.095589 8509 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:15.096576 8502 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:15.097877 8502 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:15.099076 8502 hybrid_clock.cc:648] HybridClock initialized: now 1786060935099046 us; error 58 us; skew 500 ppm
Aug 07 00:02:15 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:15.102263 8502 init.cc:377] Logged in from keytab as oryx/127.30.204.126@KRBTEST.COM (short username oryx)
I20260807 00:02:15.103432 8502 webserver.cc:460] Webserver started at http://127.30.204.126:42177/ using document root <none> and password file <none>
I20260807 00:02:15.104087 8502 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:15.104168 8502 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:15.104388 8502 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:15.106828 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "f64e620bad754807ad12eaad9ef7d37b"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "c264cfa252301352fa7f53104585095a"
server_key_iv: "69f0af7f2517f50c4672b632169bc13d"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.107482 8502 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "f64e620bad754807ad12eaad9ef7d37b"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "c264cfa252301352fa7f53104585095a"
server_key_iv: "69f0af7f2517f50c4672b632169bc13d"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.111805 8502 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:15.114758 8523 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.116427 8502 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:15.116582 8502 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "f64e620bad754807ad12eaad9ef7d37b"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "c264cfa252301352fa7f53104585095a"
server_key_iv: "69f0af7f2517f50c4672b632169bc13d"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.116720 8502 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:15.134137 8502 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:15.138558 8502 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:15.138804 8502 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:15.146773 8502 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:35533
I20260807 00:02:15.147046 8597 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:35533 every 8 connection(s)
I20260807 00:02:15.147941 8502 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:15.151082 8598 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:15.157609 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 8502
I20260807 00:02:15.157791 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:15.158110 31537 external_mini_cluster.cc:1428] Setting key e84ee588781a3978d055793a6faf2370
I20260807 00:02:15.158504 8598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: Bootstrap starting.
I20260807 00:02:15.161700 8598 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:15.162531 8598 log.cc:824] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:15.165112 8598 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: No bootstrap required, opened a new log
Aug 07 00:02:15 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:15.169787 8598 raft_consensus.cc:348] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } }
I20260807 00:02:15.170154 8598 raft_consensus.cc:374] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:15.170231 8598 raft_consensus.cc:729] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f64e620bad754807ad12eaad9ef7d37b, State: Initialized, Role: FOLLOWER
I20260807 00:02:15.170850 8598 consensus_queue.cc:260] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } }
I20260807 00:02:15.171022 8598 raft_consensus.cc:388] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:15.171099 8598 raft_consensus.cc:482] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:15.171208 8598 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:15.172276 8598 raft_consensus.cc:504] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } }
I20260807 00:02:15.172717 8598 leader_election.cc:302] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f64e620bad754807ad12eaad9ef7d37b; no voters:
I20260807 00:02:15.173118 8598 leader_election.cc:288] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:15.173259 8606 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:15.173514 8606 raft_consensus.cc:686] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 1 LEADER]: Becoming Leader. State: Replica: f64e620bad754807ad12eaad9ef7d37b, State: Running, Role: LEADER
I20260807 00:02:15.173906 8606 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } }
I20260807 00:02:15.174515 8598 sys_catalog.cc:564] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:15.175370 8606 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: New leader f64e620bad754807ad12eaad9ef7d37b. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } }
I20260807 00:02:15.175529 8606 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:15.175827 8608 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } }
I20260807 00:02:15.175926 8608 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:15.175917 8618 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:15.176040 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:15.159815 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41590 (local address 127.30.204.126:35533)
0807 00:02:15.160560 (+ 745us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:15.160577 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:15.160627 (+ 50us) server_negotiation.cc:407] Connection header received
0807 00:02:15.161408 (+ 781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:15.161432 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:15.161839 (+ 407us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:15.162221 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:15.163096 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.164548 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.165535 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.166265 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.166439 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:15.169487 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:15.169494 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:15.169514 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:15.169556 (+ 42us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:15.171739 (+ 2183us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:15.172267 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:15.172273 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:15.172279 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:15.172418 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:15.172812 (+ 394us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:15.172814 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:15.172815 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:15.173431 (+ 616us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:15.174344 (+ 913us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:15.174512 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:02:15.174863 (+ 351us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":577,"thread_start_us":254,"threads_started":1}
I20260807 00:02:15.179513 8618 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:15.185943 8618 catalog_manager.cc:1347] Generated new cluster ID: 9e4d3f548fa74b3eabbb81737eb7d398
I20260807 00:02:15.186028 8618 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:15.218598 8618 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:15.220067 8618 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:15.227066 8618 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: Generated new TSK 0
I20260807 00:02:15.227983 8618 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab.
Entry for principal oryx/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab.
I20260807 00:02:15.275194 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35533
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab
--principal=oryx/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:15.417552 8637 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:15.417866 8637 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:15.417912 8637 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:15.422238 8637 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:15.422323 8637 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:15.422390 8637 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:15.428131 8637 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab
--principal=oryx/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8637
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:15.429384 8637 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:15.430572 8637 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:15.440937 8649 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:15.440939 8646 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:15.441040 8637 server_base.cc:1034] running on GCE node
W20260807 00:02:15.440939 8647 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:15.441671 8637 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:15.442391 8637 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:15.443588 8637 hybrid_clock.cc:648] HybridClock initialized: now 1786060935443551 us; error 51 us; skew 500 ppm
Aug 07 00:02:15 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:15.446949 8637 init.cc:377] Logged in from keytab as oryx/127.30.204.65@KRBTEST.COM (short username oryx)
I20260807 00:02:15.448124 8637 webserver.cc:460] Webserver started at http://127.30.204.65:39245/ using document root <none> and password file <none>
I20260807 00:02:15.448767 8637 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:15.448858 8637 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:15.449115 8637 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:15.451622 8637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "5294b2d9be91f2bfdefb5a3158f0dada"
server_key_iv: "fd31d31c8e8b39c16d8237ad29617051"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.452340 8637 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "5294b2d9be91f2bfdefb5a3158f0dada"
server_key_iv: "fd31d31c8e8b39c16d8237ad29617051"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.456869 8637 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 00:02:15.459728 8660 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.461221 8637 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:15.461375 8637 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "5294b2d9be91f2bfdefb5a3158f0dada"
server_key_iv: "fd31d31c8e8b39c16d8237ad29617051"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.461499 8637 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:15.479398 8637 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:15.483132 8637 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:15.483352 8637 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:15.484082 8637 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:15.485073 8637 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:15.485126 8637 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.485184 8637 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:15.485217 8637 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.494624 8637 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:34259
I20260807 00:02:15.494663 8836 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:34259 every 8 connection(s)
I20260807 00:02:15.495685 8637 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:15.505239 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 8637
I20260807 00:02:15.505399 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:15.505772 31537 external_mini_cluster.cc:1428] Setting key 78be98f394bbd895f4d1701b72daf0f0
Aug 07 00:02:15 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:15.511670 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:15.497734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:36981 (local address 127.30.204.126:35533)
0807 00:02:15.497915 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:15.497919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:15.498946 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 00:02:15.500502 (+ 1556us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:15.500506 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:15.500584 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:15.500673 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:15.502014 (+ 1341us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.502561 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.503848 (+ 1287us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.504092 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.504193 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:15.507496 (+ 3303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:15.507501 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:15.507504 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:15.507532 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:15.509114 (+ 1582us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:15.509731 (+ 617us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:15.509735 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:15.509737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:15.509854 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:15.510310 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:15.510313 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:15.510315 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:15.510520 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:15.510626 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:15.511339 (+ 713us) server_negotiation.cc:299] Negotiation successful
0807 00:02:15.511507 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 00:02:15.512869 8842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:15.498133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.65:36981)
0807 00:02:15.498767 (+ 634us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:15.498819 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:15.500256 (+ 1437us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:15.500883 (+ 627us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:15.500892 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:15.501308 (+ 416us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:15.501850 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:15.501865 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.502701 (+ 836us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.502704 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:15.503712 (+ 1008us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:15.503726 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.504261 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.504266 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:15.504458 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:15.505243 (+ 785us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:15.505268 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:15.507257 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:15.509280 (+ 2023us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:15.509287 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:15.509305 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:15.509628 (+ 323us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:15.509959 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:15.509963 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:15.509965 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:15.510193 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:15.510619 (+ 426us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:15.510627 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:15.511044 (+ 417us) client_negotiation.cc:769] Sending connection context
0807 00:02:15.511288 (+ 244us) client_negotiation.cc:240] Negotiation successful
0807 00:02:15.511715 (+ 427us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":219,"threads_started":1}
I20260807 00:02:15.514360 8840 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35533
I20260807 00:02:15.514726 8840 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:15.515491 8840 heartbeater.cc:511] Master 127.30.204.126:35533 requested a full tablet report, sending...
I20260807 00:02:15.517407 8550 ts_manager.cc:194] Registered new tserver with Master: 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259)
WARNING: no policy specified for oryx/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:15.518709 8550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:36981
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab.
Entry for principal oryx/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab.
I20260807 00:02:15.539142 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35533
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab
--principal=oryx/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:15.660931 8846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:15.661288 8846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:15.661353 8846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:15.665827 8846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:15.665915 8846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:15.665998 8846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:15.671890 8846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab
--principal=oryx/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.8846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:15.673173 8846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:15.674338 8846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:15.685420 8853 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:15.685415 8856 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:15.685441 8852 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:15.685688 8846 server_base.cc:1034] running on GCE node
I20260807 00:02:15.686254 8846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:15.686991 8846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:15.688212 8846 hybrid_clock.cc:648] HybridClock initialized: now 1786060935688175 us; error 47 us; skew 500 ppm
Aug 07 00:02:15 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:15.691341 8846 init.cc:377] Logged in from keytab as oryx/127.30.204.66@KRBTEST.COM (short username oryx)
I20260807 00:02:15.692358 8846 webserver.cc:460] Webserver started at http://127.30.204.66:45287/ using document root <none> and password file <none>
I20260807 00:02:15.692950 8846 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:15.693025 8846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:15.693256 8846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:15.695500 8846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "dc326dcc80db47e4987e05da45070868"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "bbc387bf5f2690d55e2d4064538915d0"
server_key_iv: "3c61c353e51fd219b6d89fbcd63464fa"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.696146 8846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "dc326dcc80db47e4987e05da45070868"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "bbc387bf5f2690d55e2d4064538915d0"
server_key_iv: "3c61c353e51fd219b6d89fbcd63464fa"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.700438 8846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:02:15.703084 8865 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.704500 8846 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:15.704640 8846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "dc326dcc80db47e4987e05da45070868"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "bbc387bf5f2690d55e2d4064538915d0"
server_key_iv: "3c61c353e51fd219b6d89fbcd63464fa"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.704763 8846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:15.751564 8846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:15.755077 8846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:15.755292 8846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:15.756011 8846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:15.756945 8846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:15.756999 8846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.757066 8846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:15.757092 8846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.766122 8846 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:43589
I20260807 00:02:15.766145 9038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:43589 every 8 connection(s)
I20260807 00:02:15.767155 8846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:15.768116 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 8846
I20260807 00:02:15.768234 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:15.768518 31537 external_mini_cluster.cc:1428] Setting key 91e9ad95750cbaff74076a4e79a33ffa
Aug 07 00:02:15 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for oryx/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.67@KRBTEST.COM" created.
I20260807 00:02:15.784032 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:15.770048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:59209 (local address 127.30.204.126:35533)
0807 00:02:15.770195 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:15.770199 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:15.771411 (+ 1212us) server_negotiation.cc:407] Connection header received
0807 00:02:15.772940 (+ 1529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:15.772944 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:15.773005 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:15.773080 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:15.774413 (+ 1333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.775014 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.776504 (+ 1490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.776902 (+ 398us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.777027 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:15.779855 (+ 2828us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:15.779859 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:15.779861 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:15.779894 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:15.781374 (+ 1480us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:15.782086 (+ 712us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:15.782091 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:15.782093 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:15.782222 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:15.782678 (+ 456us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:15.782682 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:15.782684 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:15.782945 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:15.783052 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:15.783710 (+ 658us) server_negotiation.cc:299] Negotiation successful
0807 00:02:15.783860 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:02:15.784927 9042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:15.770628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.66:59209)
0807 00:02:15.771238 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:15.771292 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:15.772705 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:15.773294 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:15.773302 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:15.773717 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:15.774249 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:15.774264 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.775154 (+ 890us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.775158 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:15.776363 (+ 1205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:15.776376 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:15.777077 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:15.777080 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:15.777259 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:15.777926 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:15.777943 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:15.779650 (+ 1707us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:15.781522 (+ 1872us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:15.781528 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:15.781549 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:15.781925 (+ 376us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:15.782343 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:15.782346 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:15.782348 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:15.782533 (+ 185us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:15.783095 (+ 562us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:15.783101 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:15.783390 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 00:02:15.783618 (+ 228us) client_negotiation.cc:240] Negotiation successful
0807 00:02:15.783941 (+ 323us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":384,"thread_start_us":175,"threads_started":1}
I20260807 00:02:15.786209 9039 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35533
I20260807 00:02:15.786473 9039 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:15.787025 9039 heartbeater.cc:511] Master 127.30.204.126:35533 requested a full tablet report, sending...
I20260807 00:02:15.788261 8550 ts_manager.cc:194] Registered new tserver with Master: dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
I20260807 00:02:15.788851 8550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:59209
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab.
Entry for principal oryx/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab.
I20260807 00:02:15.799857 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35533
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab
--principal=oryx/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:15.920583 9047 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:15.920959 9047 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:15.921025 9047 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:15.925290 9047 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:15.925377 9047 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:15.925467 9047 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:15.931118 9047 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab
--principal=oryx/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35533
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9047
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:15.932386 9047 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:15.933504 9047 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:15.943792 9056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:15.943948 9047 server_base.cc:1034] running on GCE node
W20260807 00:02:15.943792 9058 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:15.944190 9063 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:15.944648 9047 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:15.945322 9047 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:15.946509 9047 hybrid_clock.cc:648] HybridClock initialized: now 1786060935946485 us; error 36 us; skew 500 ppm
Aug 07 00:02:15 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:15.949481 9047 init.cc:377] Logged in from keytab as oryx/127.30.204.67@KRBTEST.COM (short username oryx)
I20260807 00:02:15.950654 9047 webserver.cc:460] Webserver started at http://127.30.204.67:46097/ using document root <none> and password file <none>
I20260807 00:02:15.951349 9047 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:15.951435 9047 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:15.951741 9047 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:15.954532 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "d96f92e64db74123b48da2b979cf8c85"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "0ab191587724ae0b423d1ad289c50b38"
server_key_iv: "108667508630ee3321a306c80bf934e8"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.955125 9047 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "d96f92e64db74123b48da2b979cf8c85"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "0ab191587724ae0b423d1ad289c50b38"
server_key_iv: "108667508630ee3321a306c80bf934e8"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.959357 9047 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 00:02:15.962103 9075 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.963809 9047 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:15.963948 9047 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "d96f92e64db74123b48da2b979cf8c85"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "0ab191587724ae0b423d1ad289c50b38"
server_key_iv: "108667508630ee3321a306c80bf934e8"
server_key_version: "encryptionkey@0"
I20260807 00:02:15.964061 9047 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:15.977599 9047 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:15.981379 9047 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:15.981600 9047 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:15.982282 9047 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:15.983268 9047 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:15.983320 9047 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.983390 9047 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:15.983417 9047 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:15.992826 9047 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:43509
I20260807 00:02:15.992868 9250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:43509 every 8 connection(s)
I20260807 00:02:15.993865 9047 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:15.998320 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 9047
I20260807 00:02:15.998440 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:15.998816 31537 external_mini_cluster.cc:1428] Setting key 209bbb725d0e8421681730f8a3ef2112
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:16.010066 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:15.996036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:34441 (local address 127.30.204.126:35533)
0807 00:02:15.996238 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:15.996243 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:15.997306 (+ 1063us) server_negotiation.cc:407] Connection header received
0807 00:02:15.998992 (+ 1686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:15.998996 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:15.999062 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:15.999163 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.000813 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.001493 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.002820 (+ 1327us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.003058 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.003151 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.006017 (+ 2866us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.006021 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.006023 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.006054 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.007722 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.008414 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.008417 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.008418 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.008517 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.008889 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.008891 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.008892 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.009091 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.009188 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.009786 (+ 598us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.009923 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 00:02:16.010955 9254 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:15.996486 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.67:34441)
0807 00:02:15.997112 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:15.997173 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:15.998714 (+ 1541us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:15.999486 (+ 772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:15.999499 (+ 13us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:15.999974 (+ 475us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.000640 (+ 666us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.000658 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.001671 (+ 1013us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.001675 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.002659 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.002672 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.003279 (+ 607us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.003283 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.003517 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.004155 (+ 638us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.004175 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.005772 (+ 1597us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.007889 (+ 2117us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.007896 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.007913 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.008259 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.008634 (+ 375us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.008637 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.008639 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.008813 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.009183 (+ 370us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.009189 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.009494 (+ 305us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.009697 (+ 203us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.009998 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":340,"thread_start_us":141,"threads_started":1}
I20260807 00:02:16.012486 9251 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35533
I20260807 00:02:16.012792 9251 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:16.013425 9251 heartbeater.cc:511] Master 127.30.204.126:35533 requested a full tablet report, sending...
I20260807 00:02:16.014696 8550 ts_manager.cc:194] Registered new tserver with Master: d96f92e64db74123b48da2b979cf8c85 (127.30.204.67:43509)
I20260807 00:02:16.015317 8550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:34441
I20260807 00:02:16.025964 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.204.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.125@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.125.keytab.
Entry for principal oryx/127.30.204.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.125.keytab.
I20260807 00:02:16.058243 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:45769
--webserver_interface=127.30.204.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.125.keytab
--principal=oryx/127.30.204.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:35533,127.30.204.125:45769
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:16.180725 9259 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:16.181063 9259 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:16.181135 9259 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:16.185505 9259 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:16.185591 9259 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:16.185619 9259 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:16.185642 9259 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:16.185662 9259 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:16.191586 9259 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:35533,127.30.204.125:45769
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.125.keytab
--principal=oryx/127.30.204.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:45769
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.204.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9259
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
W20260807 00:02:16.191933 9259 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.30.204.126:35533,127.30.204.125:45769'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260807 00:02:16.192942 9259 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:16.194103 9259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:16.204051 9269 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.204172 9259 server_base.cc:1034] running on GCE node
W20260807 00:02:16.204051 9271 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:16.204099 9275 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.204985 9259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:16.206532 9259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:16.207757 9259 hybrid_clock.cc:648] HybridClock initialized: now 1786060936207719 us; error 53 us; skew 500 ppm
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:16.210980 9259 init.cc:377] Logged in from keytab as oryx/127.30.204.125@KRBTEST.COM (short username oryx)
I20260807 00:02:16.212152 9259 webserver.cc:460] Webserver started at http://127.30.204.125:38373/ using document root <none> and password file <none>
I20260807 00:02:16.212762 9259 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:16.212831 9259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:16.213024 9259 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:16.215374 9259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/instance:
uuid: "20b55adeb95b4d849c8d2cab53d50fb7"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "35f3146b7aa453cf51031ec8ab65ec7e"
server_key_iv: "b11f984d73c3539697d55e6808a5e432"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.216086 9259 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal/instance:
uuid: "20b55adeb95b4d849c8d2cab53d50fb7"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "35f3146b7aa453cf51031ec8ab65ec7e"
server_key_iv: "b11f984d73c3539697d55e6808a5e432"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.220886 9259 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 00:02:16.224292 9286 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:16.226491 9259 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:02:16.226681 9259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
uuid: "20b55adeb95b4d849c8d2cab53d50fb7"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "35f3146b7aa453cf51031ec8ab65ec7e"
server_key_iv: "b11f984d73c3539697d55e6808a5e432"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.226821 9259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:16.244573 9259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:16.248214 9259 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:16.248399 9259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:16.255831 9259 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.125:45769
I20260807 00:02:16.256117 9365 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.125:45769 every 8 connection(s)
I20260807 00:02:16.256946 9259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
I20260807 00:02:16.260073 9367 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:16.264392 9367 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:02:16.266870 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 9259
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.125@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.204.125@KRBTEST.COM
I20260807 00:02:16.279177 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.266670 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41598 (local address 127.30.204.126:35533)
0807 00:02:16.266844 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.266849 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.267971 (+ 1122us) server_negotiation.cc:407] Connection header received
0807 00:02:16.269212 (+ 1241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.269215 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.269277 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.269373 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.270404 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.270961 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.272098 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.272325 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.272424 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.275105 (+ 2681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.275109 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.275112 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.275145 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.276842 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.277497 (+ 655us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.277500 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.277501 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.277628 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.277995 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.277998 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.278000 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.278252 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.278345 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.278868 (+ 523us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.279003 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:02:16.280248 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.267793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43002 (local address 127.30.204.125:45769)
0807 00:02:16.268323 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.268334 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.268376 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:02:16.268958 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.268986 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.269274 (+ 288us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.269470 (+ 196us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.270343 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.271263 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.272142 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.272534 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.272659 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.275564 (+ 2905us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.275570 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.275584 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.275607 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.277377 (+ 1770us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.277856 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.277862 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.277863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.277944 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.278213 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.278216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.278218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.278623 (+ 405us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.278831 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.279047 (+ 216us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.279264 (+ 217us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":50,"server-negotiator.queue_time_us":436,"spinlock_wait_cycles":2944,"thread_start_us":164,"threads_started":1}
I20260807 00:02:16.280256 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.267156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:41598)
0807 00:02:16.267784 (+ 628us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.267851 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.268953 (+ 1102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.269463 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.269470 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.269899 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.270230 (+ 331us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.270247 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.271095 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.271097 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.271967 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.271977 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.272484 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.272488 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.272654 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.273249 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.273266 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.274919 (+ 1653us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.276980 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.276989 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.277001 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.277386 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.277736 (+ 350us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.277740 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.277742 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.277903 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.278361 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.278369 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.278639 (+ 270us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.278803 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.279194 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":3,"spinlock_wait_cycles":6272,"thread_start_us":145,"threads_started":1}
I20260807 00:02:16.281970 9367 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.125@KRBTEST.COM for oryx/127.30.204.125@KRBTEST.COM
I20260807 00:02:16.290567 9371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.282580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.0.0.1:43004)
0807 00:02:16.282755 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.282769 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.282845 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.283207 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.283209 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.283384 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.283549 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.283556 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.284285 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.284287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.284846 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.284854 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.285291 (+ 437us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.285293 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.285399 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.285907 (+ 508us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.285919 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.287247 (+ 1328us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.289092 (+ 1845us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.289095 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.289096 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.289316 (+ 220us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.289568 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.289571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.289573 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.289656 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.290080 (+ 424us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.290082 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.290166 (+ 84us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.290241 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.290384 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":69,"mutex_wait_us":1}
I20260807 00:02:16.290601 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.282918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43004 (local address 127.30.204.125:45769)
0807 00:02:16.283025 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.283028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.283039 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:02:16.283069 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.283071 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.283105 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.283173 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.283646 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.284177 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.284954 (+ 777us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.285202 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.285284 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.287417 (+ 2133us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.287421 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.287423 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.287446 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.288958 (+ 1512us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.289408 (+ 450us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.289411 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.289412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.289490 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.289766 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.289769 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.289770 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.289988 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.290090 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.290301 (+ 211us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.290445 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":47,"server-negotiator.queue_time_us":39}
I20260807 00:02:16.294101 9367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Bootstrap starting.
I20260807 00:02:16.296814 9367 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:16.297641 9367 log.cc:824] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:16.299957 9367 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: No bootstrap required, opened a new log
I20260807 00:02:16.303133 9367 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } }
I20260807 00:02:16.303382 9367 raft_consensus.cc:374] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:16.303428 9367 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 20b55adeb95b4d849c8d2cab53d50fb7, State: Initialized, Role: FOLLOWER
I20260807 00:02:16.303942 9367 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } }
I20260807 00:02:16.304517 9377 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } } }
I20260807 00:02:16.304654 9377 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:02:16.305310 9367 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 00:02:16.310353 9388 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:02:16.310467 9388 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:02:16.312965 9259 master_runner.cc:428] Detected that this master 20b55adeb95b4d849c8d2cab53d50fb7 is joining an existing cluster
I20260807 00:02:16.313056 9259 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.204.125:45769
I20260807 00:02:16.324232 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.316096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43018 (local address 127.30.204.125:45769)
0807 00:02:16.316223 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.316228 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.316599 (+ 371us) server_negotiation.cc:407] Connection header received
0807 00:02:16.316732 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.316735 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.316767 (+ 32us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.316828 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.317268 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.318007 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.318987 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.319206 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.319292 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.321153 (+ 1861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.321156 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.321159 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.321182 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.322618 (+ 1436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.323013 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.323016 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.323018 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.323100 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.323344 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.323347 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.323349 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.323595 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.323681 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.323926 (+ 245us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.324094 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":57,"server-negotiator.queue_time_us":50}
I20260807 00:02:16.324246 9399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.315714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.0.0.1:43018)
0807 00:02:16.316522 (+ 808us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.316535 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.316648 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.316850 (+ 202us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.316853 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.317053 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.317157 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.317165 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.318132 (+ 967us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.318135 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.318896 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.318903 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.319318 (+ 415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.319321 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.319401 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.320562 (+ 1161us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.320576 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.321006 (+ 430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.322722 (+ 1716us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.322725 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.322728 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.322919 (+ 191us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.323200 (+ 281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.323202 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.323203 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.323282 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.323712 (+ 430us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.323714 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.323789 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.323869 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.324030 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":745,"spinlock_wait_cycles":18688,"thread_start_us":108,"threads_started":1}
I20260807 00:02:16.324613 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.315773 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41602 (local address 127.30.204.126:35533)
0807 00:02:16.315905 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.315909 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.316261 (+ 352us) server_negotiation.cc:407] Connection header received
0807 00:02:16.316385 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.316388 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.316444 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.316526 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.317157 (+ 631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.317724 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.318853 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.319252 (+ 399us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.319361 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.321135 (+ 1774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.321139 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.321142 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.321174 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.322869 (+ 1695us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.323289 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.323292 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.323294 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.323385 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.323667 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.323670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.323672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.323896 (+ 224us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.323989 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.324326 (+ 337us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.324456 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 00:02:16.325112 9398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.315691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:41602)
0807 00:02:16.316161 (+ 470us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.316176 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.316287 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.316568 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.316571 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.316834 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.317006 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.317014 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.317829 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.317832 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.318718 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.318728 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.319428 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.319431 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.319534 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.320198 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.320216 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.320856 (+ 640us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.322985 (+ 2129us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.322988 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.322991 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.323205 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.323495 (+ 290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.323498 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.323500 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.323576 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.324037 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.324050 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.324146 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.324771 (+ 625us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.324921 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":382,"mutex_wait_us":1,"thread_start_us":114,"threads_started":1}
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.126@KRBTEST.COM for oryx/127.30.204.125@KRBTEST.COM
I20260807 00:02:16.339959 9374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.330035 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.30.204.125:45769)
0807 00:02:16.330155 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.330160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.330545 (+ 385us) server_negotiation.cc:407] Connection header received
0807 00:02:16.330856 (+ 311us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.330859 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.330910 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.331015 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.331817 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.332344 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.333300 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.333649 (+ 349us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.333758 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.336227 (+ 2469us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.336230 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.336232 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.336270 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.337969 (+ 1699us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.338576 (+ 607us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.338578 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.338579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.338659 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.338933 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.338937 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.338939 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.339152 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.339242 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.339653 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.339784 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 00:02:16.339993 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.329992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.0.0.1:43034)
0807 00:02:16.330415 (+ 423us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.330463 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.330613 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.331067 (+ 454us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.331074 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.331566 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.331697 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.331704 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.332460 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.332464 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.333172 (+ 708us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.333184 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.333772 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.333776 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.333904 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.334520 (+ 616us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.334533 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.336068 (+ 1535us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.338113 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.338126 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.338134 (+ 8us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.338460 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.338751 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.338754 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.338756 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.338846 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.339262 (+ 416us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.339271 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.339406 (+ 135us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.339587 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.339833 (+ 246us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":122,"threads_started":1}
I20260807 00:02:16.341717 8550 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.204.125:45769: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7"
member_type: NON_VOTER
last_known_addr {
host: "127.30.204.125"
port: 45769
}
attrs {
promote: true
}
}
dest_uuid: "f64e620bad754807ad12eaad9ef7d37b"
cas_config_opid_index: -1
I20260807 00:02:16.342499 8550 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } }
I20260807 00:02:16.344638 9404 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER 20b55adeb95b4d849c8d2cab53d50fb7 (127.30.204.125) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } }
WARNING: no policy specified for oryx/127.30.204.124@KRBTEST.COM; defaulting to no policy
I20260807 00:02:16.345431 9321 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:16.347662 9404 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.204.125:45769
I20260807 00:02:16.347760 9321 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Refusing update from remote peer f64e620bad754807ad12eaad9ef7d37b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.124@KRBTEST.COM" created.
I20260807 00:02:16.348300 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } }
I20260807 00:02:16.348407 9259 master.cc:537] Master@127.30.204.125:45769 shutting down...
I20260807 00:02:16.348423 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:16.349378 9407 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260807 00:02:16.352334 8529 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260807 00:02:16.353446 8529 consensus_peers.cc:487] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b -> Peer 20b55adeb95b4d849c8d2cab53d50fb7 (127.30.204.125:45769): Couldn't send request to peer 20b55adeb95b4d849c8d2cab53d50fb7. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 00:02:16.354143 9259 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:16.354801 9259 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:16.354977 9259 tablet_replica.cc:330] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: stopping tablet replica
I20260807 00:02:16.367069 9259 master.cc:552] Master@127.30.204.125:45769 shutdown complete.
I20260807 00:02:16.367142 9259 master_runner.cc:305] Clearing existing system tablet
I20260807 00:02:16.367401 9259 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:16.367492 9259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:16.369742 9259 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260807 00:02:16.370803 9417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:16.371266 9259 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:16.371416 9259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:16.371488 9259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
uuid: "20b55adeb95b4d849c8d2cab53d50fb7"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "35f3146b7aa453cf51031ec8ab65ec7e"
server_key_iv: "b11f984d73c3539697d55e6808a5e432"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.374183 9259 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Deleting tablet data with delete state TABLET_DATA_DELETED
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.124.keytab.
Entry for principal oryx/127.30.204.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.124.keytab.
I20260807 00:02:16.375820 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:46519
--webserver_interface=127.30.204.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.124.keytab
--principal=oryx/127.30.204.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:02:16.377063 9259 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 00:02:16.377174 9259 log.cc:1196] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260807 00:02:16.377424 9259 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Deleting consensus metadata
I20260807 00:02:16.377761 9259 master_runner.cc:315] Copying system tablet from 127.30.204.126:35533
I20260807 00:02:16.379150 9259 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: tablet copy: Beginning tablet copy session from remote peer at address 127.30.204.126:35533
I20260807 00:02:16.386982 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.379898 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41616 (local address 127.30.204.126:35533)
0807 00:02:16.380037 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.380042 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.380234 (+ 192us) server_negotiation.cc:407] Connection header received
0807 00:02:16.380362 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.380365 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.380416 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.380484 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.380965 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.381535 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.382380 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.382606 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.382706 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.383963 (+ 1257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.383967 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.383969 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.383997 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.385350 (+ 1353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.385770 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.385773 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.385774 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.385854 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.386115 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.386118 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.386120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.386315 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.386407 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.386691 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.386813 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:02:16.386992 9429 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.379800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:41616)
0807 00:02:16.380133 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.380145 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.380273 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.380516 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.380519 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.380723 (+ 204us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.380840 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.380846 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.381654 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.381658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.382258 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.382267 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.382718 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.382721 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.382818 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.383364 (+ 546us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.383375 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.383817 (+ 442us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.385463 (+ 1646us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.385467 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.385469 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.385676 (+ 207us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.385941 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.385944 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.385946 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.386031 (+ 85us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.386408 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.386411 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.386518 (+ 107us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.386606 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.386780 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":254,"thread_start_us":90,"threads_started":1}
I20260807 00:02:16.388041 8573 tablet_copy_service.cc:140] P f64e620bad754807ad12eaad9ef7d37b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 20b55adeb95b4d849c8d2cab53d50fb7 ({username='oryx', principal='oryx/127.30.204.125@KRBTEST.COM'} at 127.0.0.1:41616)
I20260807 00:02:16.388235 8573 tablet_copy_service.cc:161] P f64e620bad754807ad12eaad9ef7d37b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 20b55adeb95b4d849c8d2cab53d50fb7 at {username='oryx', principal='oryx/127.30.204.125@KRBTEST.COM'} at 127.0.0.1:41616: session id = 20b55adeb95b4d849c8d2cab53d50fb7-00000000000000000000000000000000
I20260807 00:02:16.389863 8573 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 00:02:16.391812 9259 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 00:02:16.394155 9259 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: tablet copy: Starting download of 0 data blocks...
I20260807 00:02:16.394373 9259 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: tablet copy: Starting download of 1 WAL segments...
I20260807 00:02:16.396412 9259 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 00:02:16.398231 8573 tablet_copy_service.cc:342] P f64e620bad754807ad12eaad9ef7d37b: Request end of tablet copy session 20b55adeb95b4d849c8d2cab53d50fb7-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.204.125@KRBTEST.COM'} at 127.0.0.1:41616
I20260807 00:02:16.398366 8573 tablet_copy_service.cc:433] P f64e620bad754807ad12eaad9ef7d37b: ending tablet copy session 20b55adeb95b4d849c8d2cab53d50fb7-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 20b55adeb95b4d849c8d2cab53d50fb7
I20260807 00:02:16.406725 9259 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:16.408710 9433 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:16.408715 9430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.413000 9259 server_base.cc:1034] running on GCE node
W20260807 00:02:16.412897 9438 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.413329 9259 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:16.413691 9259 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:16.414886 9259 hybrid_clock.cc:648] HybridClock initialized: now 1786060936414872 us; error 36 us; skew 500 ppm
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:16.416424 9259 init.cc:377] Logged in from keytab as oryx/127.30.204.125@KRBTEST.COM (short username oryx)
I20260807 00:02:16.416939 9259 webserver.cc:460] Webserver started at http://127.30.204.125:38045/ using document root <none> and password file <none>
I20260807 00:02:16.417147 9259 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:16.417192 9259 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:16.418319 9259 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:02:16.419246 9450 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:16.419502 9259 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260807 00:02:16.419580 9259 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
uuid: "20b55adeb95b4d849c8d2cab53d50fb7"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "35f3146b7aa453cf51031ec8ab65ec7e"
server_key_iv: "b11f984d73c3539697d55e6808a5e432"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.419670 9259 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:16.440768 9259 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:16.442782 9259 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:16.447764 9259 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.125:45769
I20260807 00:02:16.447774 9532 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.125:45769 every 8 connection(s)
I20260807 00:02:16.448370 9259 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
I20260807 00:02:16.450029 9534 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:02:16.450927 9534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Bootstrap starting.
I20260807 00:02:16.460055 9534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:02:16.460405 9534 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Bootstrap complete.
I20260807 00:02:16.460990 9534 raft_consensus.cc:348] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } }
I20260807 00:02:16.461128 9534 raft_consensus.cc:729] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 20b55adeb95b4d849c8d2cab53d50fb7, State: Initialized, Role: LEARNER
I20260807 00:02:16.461241 9534 consensus_queue.cc:260] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } }
I20260807 00:02:16.461612 9540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } }
I20260807 00:02:16.461762 9534 sys_catalog.cc:564] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:16.461822 9540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:02:16.465476 9558 catalog_manager.cc:1259] Loaded cluster ID: 9e4d3f548fa74b3eabbb81737eb7d398
I20260807 00:02:16.465641 9558 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: loading cluster ID for follower catalog manager: success
I20260807 00:02:16.467422 9558 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: acquiring CA information for follower catalog manager: success
I20260807 00:02:16.468849 9558 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:02:16.521893 8840 heartbeater.cc:503] Master 127.30.204.126:35533 was elected leader, sending a full tablet report...
W20260807 00:02:16.535496 9420 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:16.535825 9420 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:16.535868 9420 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:16.540094 9420 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:16.540180 9420 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:16.540198 9420 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:16.540215 9420 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:16.540228 9420 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:16.545994 9420 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.124.keytab
--principal=oryx/127.30.204.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:46519
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.204.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9420
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:16.547312 9420 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:16.548491 9420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:16.557685 9564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:16.557852 9565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:16.557890 9567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.558125 9420 server_base.cc:1034] running on GCE node
I20260807 00:02:16.558748 9420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:16.560038 9420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:16.561226 9420 hybrid_clock.cc:648] HybridClock initialized: now 1786060936561185 us; error 56 us; skew 500 ppm
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:16.564389 9420 init.cc:377] Logged in from keytab as oryx/127.30.204.124@KRBTEST.COM (short username oryx)
I20260807 00:02:16.565411 9420 webserver.cc:460] Webserver started at http://127.30.204.124:38709/ using document root <none> and password file <none>
I20260807 00:02:16.566006 9420 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:16.566076 9420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:16.566273 9420 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:16.568635 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/instance:
uuid: "4c1ec5ff506742d8a9ffaa32852aaca4"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "630e43b7b4ebb405b51c5ddb253d9ab5"
server_key_iv: "b62b1b38eb6ac43b4a89355cd50f61f9"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.569280 9420 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal/instance:
uuid: "4c1ec5ff506742d8a9ffaa32852aaca4"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "630e43b7b4ebb405b51c5ddb253d9ab5"
server_key_iv: "b62b1b38eb6ac43b4a89355cd50f61f9"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.573647 9420 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:16.576542 9577 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:16.578006 9420 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:16.578163 9420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
uuid: "4c1ec5ff506742d8a9ffaa32852aaca4"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "630e43b7b4ebb405b51c5ddb253d9ab5"
server_key_iv: "b62b1b38eb6ac43b4a89355cd50f61f9"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.578306 9420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:16.602500 9420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:16.609913 9420 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:16.610140 9420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:16.618355 9420 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.124:46519
I20260807 00:02:16.618695 9655 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.124:46519 every 8 connection(s)
I20260807 00:02:16.619429 9420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
I20260807 00:02:16.623533 9656 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:16.625686 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 9420
I20260807 00:02:16.629276 9656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.124@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060934, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.204.124@KRBTEST.COM
I20260807 00:02:16.642304 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.631538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41630 (local address 127.30.204.126:35533)
0807 00:02:16.631703 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.631708 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.632100 (+ 392us) server_negotiation.cc:407] Connection header received
0807 00:02:16.632297 (+ 197us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.632300 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.632347 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.632393 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.633304 (+ 911us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.633817 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.635007 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.635274 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.635369 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.637846 (+ 2477us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.637850 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.637852 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.637887 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.639718 (+ 1831us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.640420 (+ 702us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.640424 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.640425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.640526 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.640936 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.640939 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.640941 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.641180 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.641280 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.641985 (+ 705us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.642117 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:02:16.643628 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.631384 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:41630)
0807 00:02:16.631975 (+ 591us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.632026 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.632194 (+ 168us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.632574 (+ 380us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.632579 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.633032 (+ 453us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.633170 (+ 138us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.633178 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.633968 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.633971 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.634907 (+ 936us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.634916 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.635429 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.635433 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.635578 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.636078 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.636095 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.637651 (+ 1556us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.639861 (+ 2210us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.639873 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.639888 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.640300 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.640616 (+ 316us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.640619 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.640621 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.640832 (+ 211us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.641286 (+ 454us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.641294 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.641679 (+ 385us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.641935 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.642328 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":412,"mutex_wait_us":1,"thread_start_us":194,"threads_started":1}
I20260807 00:02:16.643626 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.627535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52684 (local address 127.30.204.124:46519)
0807 00:02:16.628136 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.628159 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.628215 (+ 56us) server_negotiation.cc:407] Connection header received
0807 00:02:16.629178 (+ 963us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.629206 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.629707 (+ 501us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.630120 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.631052 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.632549 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.633879 (+ 1330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.634643 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.634806 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.638301 (+ 3495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.638309 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.638327 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.638354 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.640427 (+ 2073us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.640909 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.640915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.640917 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.641008 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.641346 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.641349 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.641350 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.641638 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.641846 (+ 208us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.642507 (+ 661us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.642637 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":39,"server-negotiator.queue_time_us":404,"thread_start_us":172,"threads_started":1}
I20260807 00:02:16.645210 9656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.124@KRBTEST.COM for oryx/127.30.204.125@KRBTEST.COM
I20260807 00:02:16.654969 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.645867 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.30.204.125:45769)
0807 00:02:16.646160 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.646174 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.646191 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:02:16.646287 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.646290 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.646346 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.646431 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.646938 (+ 507us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.647725 (+ 787us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.648891 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.649132 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.649228 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.651660 (+ 2432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.651664 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.651666 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.651694 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.653180 (+ 1486us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.653691 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.653694 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.653695 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.653783 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.654053 (+ 270us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.654055 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.654056 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.654271 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.654351 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.654700 (+ 349us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.654829 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":101,"threads_started":1}
I20260807 00:02:16.654978 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.645745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.0.0.1:43048)
0807 00:02:16.646084 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.646099 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.646187 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.646459 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.646462 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.646670 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.646827 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.646836 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.647835 (+ 999us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.647839 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.648738 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.648751 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.649250 (+ 499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.649254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.649372 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.649883 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.649898 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.651429 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.653311 (+ 1882us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.653315 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.653318 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.653591 (+ 273us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.653869 (+ 278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.653873 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.653875 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.653974 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.654360 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.654363 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.654490 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.654606 (+ 116us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.654786 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259}
I20260807 00:02:16.655958 9656 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.124@KRBTEST.COM for oryx/127.30.204.124@KRBTEST.COM
I20260807 00:02:16.664783 9661 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.656552 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.0.0.1:52696)
0807 00:02:16.656701 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.656715 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.656801 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.657005 (+ 204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.657007 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.657182 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.657264 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.657269 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.658436 (+ 1167us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.658440 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.658957 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.658966 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.659412 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.659415 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.659547 (+ 132us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.660031 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.660044 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.661433 (+ 1389us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.663134 (+ 1701us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.663138 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.663140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.663415 (+ 275us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.663729 (+ 314us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.663731 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.663732 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.663801 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.664324 (+ 523us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.664327 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.664419 (+ 92us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.664512 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.664636 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":18}
I20260807 00:02:16.665119 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.656640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52696 (local address 127.30.204.124:46519)
0807 00:02:16.656738 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.656742 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.656756 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:16.656882 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.656885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.656923 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.656988 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.657364 (+ 376us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.658323 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.659064 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.659307 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.659397 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.661596 (+ 2199us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.661599 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.661602 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.661627 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.663020 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.663558 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.663561 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.663562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.663638 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.663886 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.663890 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.663892 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.664193 (+ 301us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.664309 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.664582 (+ 273us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.664726 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":47}
I20260807 00:02:16.669061 9656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Bootstrap starting.
I20260807 00:02:16.672438 9656 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:16.673502 9656 log.cc:824] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:16.676385 9656 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: No bootstrap required, opened a new log
I20260807 00:02:16.681079 9656 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } }
I20260807 00:02:16.681424 9656 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:16.681494 9656 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c1ec5ff506742d8a9ffaa32852aaca4, State: Initialized, Role: FOLLOWER
I20260807 00:02:16.682121 9656 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } }
I20260807 00:02:16.682787 9664 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } } }
I20260807 00:02:16.682942 9664 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:02:16.683914 9656 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: configured and running, proceeding with master startup.
W20260807 00:02:16.689550 9682 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:02:16.689756 9682 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:02:16.693797 9420 master_runner.cc:428] Detected that this master 4c1ec5ff506742d8a9ffaa32852aaca4 is joining an existing cluster
I20260807 00:02:16.693892 9420 master_runner.cc:432] Initiating AddMaster RPC to add 127.30.204.124:46519
I20260807 00:02:16.698215 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 8637
I20260807 00:02:16.711894 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.696745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43064 (local address 127.30.204.125:45769)
0807 00:02:16.696883 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.696897 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.697537 (+ 640us) server_negotiation.cc:407] Connection header received
0807 00:02:16.697733 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.697737 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.697794 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.697848 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.703396 (+ 5548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.704272 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.705736 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.706116 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.706201 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.707845 (+ 1644us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.707850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.707853 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.707883 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.709755 (+ 1872us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.710382 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.710386 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.710388 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.710486 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.710808 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.710812 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.710814 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.711052 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.711126 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.711543 (+ 417us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.711710 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:02:16.712790 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.697003 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.0.0.1:43064)
0807 00:02:16.697442 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.697456 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.697608 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.698006 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.698008 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.698230 (+ 222us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.703122 (+ 4892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.703136 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.704542 (+ 1406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.704546 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.705521 (+ 975us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.705533 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.706412 (+ 879us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.706416 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.706516 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.707110 (+ 594us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.707127 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.707665 (+ 538us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.709962 (+ 2297us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.709966 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.709968 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.710238 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.710596 (+ 358us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.710599 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.710602 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.710707 (+ 105us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.711199 (+ 492us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.711202 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.711330 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.712398 (+ 1068us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.712578 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":373,"thread_start_us":127,"threads_started":1}
I20260807 00:02:16.760172 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.696700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41638 (local address 127.30.204.126:35533)
0807 00:02:16.697042 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.697048 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.697553 (+ 505us) server_negotiation.cc:407] Connection header received
0807 00:02:16.697611 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.697614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.697671 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.697764 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.749799 (+ 52035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.750696 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.752534 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.752964 (+ 430us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.753101 (+ 137us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.755789 (+ 2688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.755795 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.755798 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.755839 (+ 41us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.757793 (+ 1954us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.758441 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.758446 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.758448 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.758565 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.758946 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.758950 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.758952 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.759241 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.759333 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.759814 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.759978 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260807 00:02:16.760787 9691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.696617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:41638)
0807 00:02:16.697333 (+ 716us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.697348 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.697445 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.697776 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.697779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.698038 (+ 259us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.698186 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.698194 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.750906 (+ 52712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.750912 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.752136 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.752160 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.753320 (+ 1160us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.753327 (+ 7us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.753449 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.754914 (+ 1465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.754932 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.755546 (+ 614us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.757953 (+ 2407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.757957 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.757960 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.758293 (+ 333us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.758698 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.758701 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.758704 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.758817 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.759419 (+ 602us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.759422 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.759564 (+ 142us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.760386 (+ 822us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.760575 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":628,"thread_start_us":114,"threads_started":1}
I20260807 00:02:16.764431 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.753812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52700 (local address 127.30.204.124:46519)
0807 00:02:16.753928 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.753932 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.755325 (+ 1393us) server_negotiation.cc:407] Connection header received
0807 00:02:16.755509 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.755512 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.755552 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.755596 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.756467 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.757561 (+ 1094us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.758642 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.758959 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.759026 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.760602 (+ 1576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.760606 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.760609 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.760648 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.762622 (+ 1974us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.763187 (+ 565us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.763190 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.763192 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.763266 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.763538 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.763541 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.763543 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.763777 (+ 234us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.763849 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.764131 (+ 282us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.764259 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 00:02:16.765223 9690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.753556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.0.0.1:52700)
0807 00:02:16.755245 (+ 1689us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.755261 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.755389 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.755735 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.755738 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.755924 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.756040 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.756047 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.757699 (+ 1652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.757703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.758521 (+ 818us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.758530 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.759169 (+ 639us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.759173 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.759268 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.759779 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.759793 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.760181 (+ 388us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.762790 (+ 2609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.762840 (+ 50us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.762842 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.763087 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.763354 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.763357 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.763359 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.763435 (+ 76us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.763904 (+ 469us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.763906 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.763985 (+ 79us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.764916 (+ 931us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.765066 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1598}
I20260807 00:02:16.781107 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.775131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.0.0.1:43078)
0807 00:02:16.775440 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.775876 (+ 436us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.775983 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.776460 (+ 477us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.776464 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.776488 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:16.776600 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.776609 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.777610 (+ 1001us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.777614 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.778902 (+ 1288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.778915 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.780607 (+ 1692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.780611 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.780728 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.780797 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.780865 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.780959 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":217}
I20260807 00:02:16.781816 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.775711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43078 (local address 127.30.204.125:45769)
0807 00:02:16.775843 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.775847 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.776067 (+ 220us) server_negotiation.cc:407] Connection header received
0807 00:02:16.776130 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.776133 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.776199 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.776304 (+ 105us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:16.776744 (+ 440us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.777477 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.779352 (+ 1875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.780065 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.780189 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.780381 (+ 192us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.781595 (+ 1214us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.781689 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:02:16.782579 9495 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 LEARNER]: Deduplicated request from leader. Original: 0.0->[1.1-1.5] Dedup: 1.5->[]
I20260807 00:02:16.783219 9540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f64e620bad754807ad12eaad9ef7d37b. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } }
I20260807 00:02:16.783370 9540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: LEARNER
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060935, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.126@KRBTEST.COM for oryx/127.30.204.124@KRBTEST.COM
I20260807 00:02:16.787797 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:34259
--local_ip_for_outbound_sockets=127.30.204.65
--tserver_master_addrs=127.30.204.126:35533
--webserver_port=39245
--webserver_interface=127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab
--principal=oryx/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:02:16.791116 9039 heartbeater.cc:503] Master 127.30.204.126:35533 was elected leader, sending a full tablet report...
I20260807 00:02:16.839901 9401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.780920 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.0.0.1:52704)
0807 00:02:16.781302 (+ 382us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.781315 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.781396 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.781833 (+ 437us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.781836 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.782039 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.782137 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.782144 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.783588 (+ 1444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.783591 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.784361 (+ 770us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.784374 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.785073 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.785077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.785159 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.785668 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.785682 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.787394 (+ 1712us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.837768 (+ 50374us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.837773 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.837776 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.838116 (+ 340us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.838522 (+ 406us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.838527 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.838529 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.838659 (+ 130us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.839227 (+ 568us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.839231 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.839359 (+ 128us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.839500 (+ 141us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.839687 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":117,"threads_started":1}
I20260807 00:02:16.840711 9660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.781301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52704 (local address 127.30.204.124:46519)
0807 00:02:16.781416 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.781421 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.781434 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:02:16.781518 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.781521 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.781582 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.781650 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.782267 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.783030 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.784503 (+ 1473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.784853 (+ 350us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.784947 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.787614 (+ 2667us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.787618 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.787620 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.787652 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.837551 (+ 49899us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.838255 (+ 704us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.838260 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.838262 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.838377 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.838784 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.838787 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.838789 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.839064 (+ 275us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.839169 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.840353 (+ 1184us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.840503 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:02:16.841881 8550 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.30.204.124:46519: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4"
member_type: NON_VOTER
last_known_addr {
host: "127.30.204.124"
port: 46519
}
attrs {
promote: true
}
}
dest_uuid: "f64e620bad754807ad12eaad9ef7d37b"
cas_config_opid_index: 5
I20260807 00:02:16.842583 8550 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } }
I20260807 00:02:16.843951 9407 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.845701 9407 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.30.204.124:46519
I20260807 00:02:16.845957 9407 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.846099 9407 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:16.846230 9420 master.cc:537] Master@127.30.204.124:46519 shutting down...
I20260807 00:02:16.847234 9495 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 LEARNER]: Refusing update from remote peer f64e620bad754807ad12eaad9ef7d37b: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260807 00:02:16.847733 9404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
W20260807 00:02:16.848593 8525 consensus_peers.cc:487] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b -> Peer 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124:46519): Couldn't send request to peer 4c1ec5ff506742d8a9ffaa32852aaca4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 00:02:16.849819 9495 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.852255 9420 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 0 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:16.852559 9420 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 0 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:16.852650 9420 tablet_replica.cc:330] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: stopping tablet replica
I20260807 00:02:16.853185 9540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: NON_VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: true } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.853314 9540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:02:16.853631 9404 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: attempting to promote NON_VOTER 20b55adeb95b4d849c8d2cab53d50fb7 to VOTER
I20260807 00:02:16.854182 9404 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } }
W20260807 00:02:16.856587 8525 consensus_peers.cc:487] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b -> Peer 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124:46519): Couldn't send request to peer 4c1ec5ff506742d8a9ffaa32852aaca4. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260807 00:02:16.857244 9493 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 LEARNER]: Refusing update from remote peer f64e620bad754807ad12eaad9ef7d37b: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260807 00:02:16.857792 9404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
I20260807 00:02:16.860445 9697 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 20b55adeb95b4d849c8d2cab53d50fb7 (127.30.204.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.862496 9697 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.862637 9697 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:16.862814 9697 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.862888 9697 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:16.863241 9420 master.cc:552] Master@127.30.204.124:46519 shutdown complete.
I20260807 00:02:16.863301 9420 master_runner.cc:305] Clearing existing system tablet
I20260807 00:02:16.863817 9495 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, 20b55adeb95b4d849c8d2cab53d50fb7 (127.30.204.125) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.864679 9420 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:16.864763 9420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:16.867267 9540 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.867436 9540 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:02:16.868247 9420 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:16.869263 9701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:16.869491 9420 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:16.869637 9420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:16.869675 9420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
uuid: "4c1ec5ff506742d8a9ffaa32852aaca4"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "630e43b7b4ebb405b51c5ddb253d9ab5"
server_key_iv: "b62b1b38eb6ac43b4a89355cd50f61f9"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.872565 9420 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:16.876003 9420 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260807 00:02:16.876255 9420 log.cc:1196] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260807 00:02:16.876818 9420 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Deleting consensus metadata
I20260807 00:02:16.877068 9420 master_runner.cc:315] Copying system tablet from 127.30.204.126:35533
I20260807 00:02:16.878501 9420 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: tablet copy: Beginning tablet copy session from remote peer at address 127.30.204.126:35533
I20260807 00:02:16.889219 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.879289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41648 (local address 127.30.204.126:35533)
0807 00:02:16.879441 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:16.879446 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:16.879737 (+ 291us) server_negotiation.cc:407] Connection header received
0807 00:02:16.879961 (+ 224us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:16.879964 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:16.880021 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:16.880079 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:16.880813 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.881717 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.883277 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.883646 (+ 369us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.883777 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.885280 (+ 1503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:16.885284 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:16.885287 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:16.885321 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:16.887235 (+ 1914us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.887814 (+ 579us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.887818 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.887820 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.887932 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:16.888236 (+ 304us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:16.888240 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:16.888242 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:16.888505 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:16.888611 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:16.888920 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 00:02:16.889057 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 00:02:16.889238 9707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:16.879207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:41648)
0807 00:02:16.879627 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:16.879646 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:16.879837 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:16.880272 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:16.880275 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:16.880516 (+ 241us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:16.880657 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.880666 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.881891 (+ 1225us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.881894 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.882836 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:16.882849 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:16.883796 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:16.883800 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:16.883894 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:16.884526 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:16.884543 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:16.885088 (+ 545us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:16.887382 (+ 2294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.887386 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.887388 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.887697 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.888026 (+ 329us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:16.888029 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:16.888031 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:16.888138 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:16.888608 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:16.888611 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:16.888731 (+ 120us) client_negotiation.cc:769] Sending connection context
0807 00:02:16.888853 (+ 122us) client_negotiation.cc:240] Negotiation successful
0807 00:02:16.889035 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":339,"thread_start_us":95,"threads_started":1}
I20260807 00:02:16.890100 8573 tablet_copy_service.cc:140] P f64e620bad754807ad12eaad9ef7d37b: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 4c1ec5ff506742d8a9ffaa32852aaca4 ({username='oryx', principal='oryx/127.30.204.124@KRBTEST.COM'} at 127.0.0.1:41648)
I20260807 00:02:16.890225 8573 tablet_copy_service.cc:161] P f64e620bad754807ad12eaad9ef7d37b: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 4c1ec5ff506742d8a9ffaa32852aaca4 at {username='oryx', principal='oryx/127.30.204.124@KRBTEST.COM'} at 127.0.0.1:41648: session id = 4c1ec5ff506742d8a9ffaa32852aaca4-00000000000000000000000000000000
I20260807 00:02:16.891552 8573 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: Tablet Copy: opened 0 blocks and 1 log segments
I20260807 00:02:16.893054 9420 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260807 00:02:16.895746 9420 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: tablet copy: Starting download of 0 data blocks...
I20260807 00:02:16.895962 9420 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: tablet copy: Starting download of 1 WAL segments...
I20260807 00:02:16.897708 9420 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260807 00:02:16.899894 8573 tablet_copy_service.cc:342] P f64e620bad754807ad12eaad9ef7d37b: Request end of tablet copy session 4c1ec5ff506742d8a9ffaa32852aaca4-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.30.204.124@KRBTEST.COM'} at 127.0.0.1:41648
I20260807 00:02:16.900012 8573 tablet_copy_service.cc:433] P f64e620bad754807ad12eaad9ef7d37b: ending tablet copy session 4c1ec5ff506742d8a9ffaa32852aaca4-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 4c1ec5ff506742d8a9ffaa32852aaca4
I20260807 00:02:16.907020 9420 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:16.908931 9715 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.909200 9420 server_base.cc:1034] running on GCE node
W20260807 00:02:16.909229 9710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:16.908974 9711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.909552 9420 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:16.909886 9420 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:16.911067 9420 hybrid_clock.cc:648] HybridClock initialized: now 1786060936911046 us; error 38 us; skew 500 ppm
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:16.912586 9420 init.cc:377] Logged in from keytab as oryx/127.30.204.124@KRBTEST.COM (short username oryx)
I20260807 00:02:16.913079 9420 webserver.cc:460] Webserver started at http://127.30.204.124:43063/ using document root <none> and password file <none>
I20260807 00:02:16.913255 9420 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:16.913291 9420 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:16.914434 9420 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:16.915370 9728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:16.915666 9420 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:16.915738 9420 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
uuid: "4c1ec5ff506742d8a9ffaa32852aaca4"
format_stamp: "Formatted at 2026-08-07 00:02:16 on dist-test-slave-pww0"
server_key: "630e43b7b4ebb405b51c5ddb253d9ab5"
server_key_iv: "b62b1b38eb6ac43b4a89355cd50f61f9"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.915833 9420 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:16.924486 9420 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:16.926342 9420 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:16.931134 9420 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.124:46519
I20260807 00:02:16.931993 9420 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
I20260807 00:02:16.932291 9808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.124:46519 every 8 connection(s)
I20260807 00:02:16.933655 9809 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:02:16.934536 9809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Bootstrap starting.
I20260807 00:02:16.944286 9809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:02:16.944630 9809 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: Bootstrap complete.
I20260807 00:02:16.945214 9809 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } }
I20260807 00:02:16.945346 9809 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 4c1ec5ff506742d8a9ffaa32852aaca4, State: Initialized, Role: LEARNER
I20260807 00:02:16.945456 9809 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } }
I20260807 00:02:16.945955 9809 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:16.945892 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:16.946040 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: This master's current role is: LEARNER
I20260807 00:02:16.948727 9831 catalog_manager.cc:1259] Loaded cluster ID: 9e4d3f548fa74b3eabbb81737eb7d398
I20260807 00:02:16.948794 9831 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: loading cluster ID for follower catalog manager: success
I20260807 00:02:16.951000 9831 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: acquiring CA information for follower catalog manager: success
W20260807 00:02:16.951855 9696 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:16.952215 9696 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:16.952286 9696 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 00:02:16.952620 9831 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
W20260807 00:02:16.957743 9696 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:16.957829 9696 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:16.957893 9696 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:16.964212 9696 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab
--principal=oryx/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:34259
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=39245
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.9696
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:16.965498 9696 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:16.966600 9696 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:16.977500 9839 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:16.977514 9835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.977653 9696 server_base.cc:1034] running on GCE node
W20260807 00:02:16.977495 9837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:16.978410 9696 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:16.979281 9696 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:16.980494 9696 hybrid_clock.cc:648] HybridClock initialized: now 1786060936980462 us; error 67 us; skew 500 ppm
Aug 07 00:02:16 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:16.983729 9696 init.cc:377] Logged in from keytab as oryx/127.30.204.65@KRBTEST.COM (short username oryx)
I20260807 00:02:16.984771 9696 webserver.cc:460] Webserver started at http://127.30.204.65:39245/ using document root <none> and password file <none>
I20260807 00:02:16.985387 9696 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:16.985491 9696 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:16.989935 9696 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:16.992405 9850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:16.994010 9696 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:16.994136 9696 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "5294b2d9be91f2bfdefb5a3158f0dada"
server_key_iv: "fd31d31c8e8b39c16d8237ad29617051"
server_key_version: "encryptionkey@0"
I20260807 00:02:16.994735 9696 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:17.015604 9696 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:17.017733 9251 heartbeater.cc:503] Master 127.30.204.126:35533 was elected leader, sending a full tablet report...
I20260807 00:02:17.019562 9696 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:17.019758 9696 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:17.020414 9696 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:17.021420 9696 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:17.021472 9696 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.021513 9696 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:17.021533 9696 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.030652 9696 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:34259
I20260807 00:02:17.030718 10024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:34259 every 8 connection(s)
I20260807 00:02:17.031824 9696 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:17.039695 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 9696
I20260807 00:02:17.039908 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 8846
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.125@KRBTEST.COM
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060936, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.124@KRBTEST.COM
I20260807 00:02:17.050621 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.034015 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:48251 (local address 127.30.204.125:45769)
0807 00:02:17.034178 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.034184 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.035725 (+ 1541us) server_negotiation.cc:407] Connection header received
0807 00:02:17.036634 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.036639 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.036706 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.036763 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.038323 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.038870 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.040911 (+ 2041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.041279 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.041363 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.045661 (+ 4298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.045666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.045669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.045702 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.047772 (+ 2070us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.048540 (+ 768us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.048545 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.048546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.048664 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.049095 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.049098 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.049100 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.049355 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.049440 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.050295 (+ 855us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.050445 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:02:17.050678 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.034530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:50549 (local address 127.30.204.126:35533)
0807 00:02:17.034671 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.034675 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.035929 (+ 1254us) server_negotiation.cc:407] Connection header received
0807 00:02:17.036635 (+ 706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.036667 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.036722 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.036771 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.038341 (+ 1570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.038911 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.040912 (+ 2001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.041279 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.041363 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.045716 (+ 4353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.045722 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.045725 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.045764 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.047895 (+ 2131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.048632 (+ 737us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.048636 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.048639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.048764 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.049131 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.049135 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.049137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.049401 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.049493 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.050326 (+ 833us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.050484 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 00:02:17.051568 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.034908 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:48915 (local address 127.30.204.124:46519)
0807 00:02:17.035599 (+ 691us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.035605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.035625 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:02:17.036615 (+ 990us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.036619 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.036693 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.036775 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.039026 (+ 2251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.039930 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.041443 (+ 1513us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.041937 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.042068 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.045661 (+ 3593us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.045666 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.045669 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.045702 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.048569 (+ 2867us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.049049 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.049053 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.049055 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.049171 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.049462 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.049466 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.049467 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.049689 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.049781 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.051235 (+ 1454us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.051379 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":605,"thread_start_us":83,"threads_started":1}
I20260807 00:02:17.052239 10029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.034377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.30.204.65:48251)
0807 00:02:17.035595 (+ 1218us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.035609 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.036397 (+ 788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.036988 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.036997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.037525 (+ 528us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.038157 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.038167 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.039039 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.039043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.040745 (+ 1702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.040759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.041609 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.041613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.041917 (+ 304us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.042911 (+ 994us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.042935 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.045404 (+ 2469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.047911 (+ 2507us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.047920 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.047943 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.048393 (+ 450us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.048774 (+ 381us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.048777 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.048779 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.048988 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.049506 (+ 518us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.049515 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.049949 (+ 434us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.050816 (+ 867us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.050978 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1123,"mutex_wait_us":289,"spinlock_wait_cycles":53120,"thread_start_us":165,"threads_started":1}
I20260807 00:02:17.052243 10030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.034377 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.65:50549)
0807 00:02:17.035823 (+ 1446us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.035836 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.036397 (+ 561us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.036988 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.036997 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.037524 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.038159 (+ 635us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.038167 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.039047 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.039050 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.040745 (+ 1695us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.040759 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.041609 (+ 850us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.041613 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.041918 (+ 305us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.042687 (+ 769us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.042714 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.045393 (+ 2679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.048137 (+ 2744us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.048142 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.048145 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.048482 (+ 337us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.048891 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.048894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.048896 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.049009 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.049560 (+ 551us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.049563 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.049954 (+ 391us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.050856 (+ 902us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.051065 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1356,"mutex_wait_us":491,"spinlock_wait_cycles":42624,"thread_start_us":102,"threads_started":1}
I20260807 00:02:17.053025 10031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.034376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.30.204.65:48915)
0807 00:02:17.035342 (+ 966us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.035405 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.036387 (+ 982us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.036987 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.036995 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.037522 (+ 527us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.038142 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.038157 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.040082 (+ 1925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.040085 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.041137 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.041151 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.042265 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.042270 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.042397 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.042914 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.042933 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.045432 (+ 2499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.048691 (+ 3259us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.048694 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.048696 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.048931 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.049280 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.049283 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.049285 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.049366 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.049824 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.049827 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.049949 (+ 122us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.050208 (+ 259us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.050612 (+ 404us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":704,"mutex_wait_us":31,"spinlock_wait_cycles":24832,"thread_start_us":157,"threads_started":1}
I20260807 00:02:17.053862 10027 heartbeater.cc:347] Connected to a master server at 127.30.204.125:45769
I20260807 00:02:17.054011 10025 heartbeater.cc:347] Connected to a master server at 127.30.204.124:46519
I20260807 00:02:17.054136 10025 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.054136 10027 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.054813 10025 heartbeater.cc:511] Master 127.30.204.124:46519 requested a full tablet report, sending...
I20260807 00:02:17.054811 10027 heartbeater.cc:511] Master 127.30.204.125:45769 requested a full tablet report, sending...
I20260807 00:02:17.056699 9476 ts_manager.cc:194] Registered new tserver with Master: 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259)
I20260807 00:02:17.056689 9756 ts_manager.cc:194] Registered new tserver with Master: 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259)
I20260807 00:02:17.083315 10026 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35533
I20260807 00:02:17.083520 10026 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.083814 10026 heartbeater.cc:511] Master 127.30.204.126:35533 requested a full tablet report, sending...
I20260807 00:02:17.084913 8550 ts_manager.cc:194] Re-registered known tserver with Master: 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259)
I20260807 00:02:17.085672 8550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:50549
I20260807 00:02:17.088804 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:43589
--local_ip_for_outbound_sockets=127.30.204.66
--tserver_master_addrs=127.30.204.126:35533
--webserver_port=45287
--webserver_interface=127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab
--principal=oryx/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:17.221767 10036 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:17.222071 10036 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:17.222124 10036 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:17.226267 10036 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:17.226347 10036 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:17.226409 10036 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:17.232052 10036 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab
--principal=oryx/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:43589
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=45287
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10036
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:17.233289 10036 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:17.234417 10036 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:17.244683 10041 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:17.244688 10045 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:17.244772 10036 server_base.cc:1034] running on GCE node
W20260807 00:02:17.244705 10043 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:17.245515 10036 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:17.246264 10036 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:17.247448 10036 hybrid_clock.cc:648] HybridClock initialized: now 1786060937247419 us; error 46 us; skew 500 ppm
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:17.250490 10036 init.cc:377] Logged in from keytab as oryx/127.30.204.66@KRBTEST.COM (short username oryx)
I20260807 00:02:17.251502 10036 webserver.cc:460] Webserver started at http://127.30.204.66:45287/ using document root <none> and password file <none>
I20260807 00:02:17.252147 10036 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:17.252233 10036 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:17.256539 10036 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:02:17.259102 10060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.260582 10036 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:17.260711 10036 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "dc326dcc80db47e4987e05da45070868"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "bbc387bf5f2690d55e2d4064538915d0"
server_key_iv: "3c61c353e51fd219b6d89fbcd63464fa"
server_key_version: "encryptionkey@0"
I20260807 00:02:17.261358 10036 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:17.293648 10036 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:17.297205 10036 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:17.297402 10036 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:17.298103 10036 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:17.299108 10036 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:17.299170 10036 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.299235 10036 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:17.299259 10036 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.308761 10036 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:43589
I20260807 00:02:17.308804 10234 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:43589 every 8 connection(s)
I20260807 00:02:17.309870 10036 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:17.319001 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 10036
I20260807 00:02:17.319244 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 9047
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.124@KRBTEST.COM
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.125@KRBTEST.COM
I20260807 00:02:17.325762 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.312080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:51189 (local address 127.30.204.124:46519)
0807 00:02:17.312385 (+ 305us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.312389 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.313210 (+ 821us) server_negotiation.cc:407] Connection header received
0807 00:02:17.314611 (+ 1401us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.314614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.314668 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.314737 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.316109 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.316975 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318171 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.318456 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318552 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.321532 (+ 2980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.321536 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.321537 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.321566 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.323068 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.323673 (+ 605us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.323677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.323678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.323760 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.324107 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.324110 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.324111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.324321 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.324390 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.325264 (+ 874us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.325413 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218}
I20260807 00:02:17.327440 10247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.312484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.30.204.66:51189)
0807 00:02:17.313037 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.313093 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.314089 (+ 996us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.314765 (+ 676us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.314768 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.315261 (+ 493us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.315973 (+ 712us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.315990 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.317138 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.317140 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.318043 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.318053 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.318608 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318613 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.318809 (+ 196us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.319633 (+ 824us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.319656 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.321334 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.323222 (+ 1888us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.323229 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.323247 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.323573 (+ 326us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.323840 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.323842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.323844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.324009 (+ 165us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.324438 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.324445 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.324767 (+ 322us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.326003 (+ 1236us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.326391 (+ 388us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"mutex_wait_us":143,"spinlock_wait_cycles":2816,"thread_start_us":144,"threads_started":1}
I20260807 00:02:17.329406 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.312116 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:34699 (local address 127.30.204.125:45769)
0807 00:02:17.312490 (+ 374us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.312495 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.314631 (+ 2136us) server_negotiation.cc:407] Connection header received
0807 00:02:17.314688 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.314691 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.314743 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.314826 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.316166 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.316963 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318346 (+ 1383us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.318651 (+ 305us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318776 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.324633 (+ 5857us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.324638 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.324642 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.324690 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.326377 (+ 1687us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.327270 (+ 893us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.327276 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.327278 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.327395 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.328260 (+ 865us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.328266 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.328268 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.328600 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.328692 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.329141 (+ 449us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.329268 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":293}
I20260807 00:02:17.329838 10248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.312484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.30.204.66:34699)
0807 00:02:17.314489 (+ 2005us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.314502 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.314596 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.314956 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.314959 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.315269 (+ 310us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.315979 (+ 710us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.315995 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.317079 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.317082 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.318217 (+ 1135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.318228 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.318924 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318928 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.319017 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.322516 (+ 3499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.322533 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.324417 (+ 1884us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.326822 (+ 2405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.326826 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.326829 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.327121 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.327968 (+ 847us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.327972 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.327974 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.328110 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.328782 (+ 672us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.328785 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.328910 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.329030 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.329229 (+ 199us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1910,"mutex_wait_us":273,"spinlock_wait_cycles":12544,"thread_start_us":108,"threads_started":1}
I20260807 00:02:17.330574 10239 heartbeater.cc:347] Connected to a master server at 127.30.204.125:45769
I20260807 00:02:17.330845 10239 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.331431 10239 heartbeater.cc:511] Master 127.30.204.125:45769 requested a full tablet report, sending...
I20260807 00:02:17.332891 9476 ts_manager.cc:194] Registered new tserver with Master: dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
I20260807 00:02:17.336925 10235 heartbeater.cc:347] Connected to a master server at 127.30.204.124:46519
I20260807 00:02:17.337095 10235 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.337354 10235 heartbeater.cc:511] Master 127.30.204.124:46519 requested a full tablet report, sending...
I20260807 00:02:17.338330 9756 ts_manager.cc:194] Registered new tserver with Master: dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:17.342545 10250 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.312485 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.66:50943)
0807 00:02:17.313575 (+ 1090us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.313588 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.314108 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.314648 (+ 540us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.314658 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.315260 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.315979 (+ 719us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.315995 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.316890 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.316892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.318221 (+ 1329us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.318228 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.318876 (+ 648us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318880 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.318961 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.337241 (+ 18280us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.337259 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.338935 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.341019 (+ 2084us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.341023 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.341025 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.341249 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.341553 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.341555 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.341556 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.341627 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.342060 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.342062 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.342162 (+ 100us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.342249 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.342379 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":982,"mutex_wait_us":205,"spinlock_wait_cycles":33664,"thread_start_us":133,"threads_started":1}
I20260807 00:02:17.342625 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.311966 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:50943 (local address 127.30.204.126:35533)
0807 00:02:17.312268 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.312273 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.313658 (+ 1385us) server_negotiation.cc:407] Connection header received
0807 00:02:17.314308 (+ 650us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.314312 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.314378 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.314460 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.316157 (+ 1697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.316738 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318366 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.318646 (+ 280us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.318750 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.339144 (+ 20394us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.339148 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.339150 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.339185 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.340860 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.341355 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.341359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.341361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.341468 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.341717 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.341720 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.341722 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.341962 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.342061 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.342321 (+ 260us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.342459 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202}
I20260807 00:02:17.343748 10236 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35533
I20260807 00:02:17.343868 10236 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.344046 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:43509
--local_ip_for_outbound_sockets=127.30.204.67
--tserver_master_addrs=127.30.204.126:35533
--webserver_port=46097
--webserver_interface=127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab
--principal=oryx/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42613
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:02:17.344075 10236 heartbeater.cc:511] Master 127.30.204.126:35533 requested a full tablet report, sending...
I20260807 00:02:17.344878 8550 ts_manager.cc:194] Re-registered known tserver with Master: dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
I20260807 00:02:17.345485 8550 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:50943
I20260807 00:02:17.377604 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.373740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52716 (local address 127.30.204.124:46519)
0807 00:02:17.373882 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.373888 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.374116 (+ 228us) server_negotiation.cc:407] Connection header received
0807 00:02:17.374270 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.374274 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.374339 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.374457 (+ 118us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:17.374731 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.375278 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.376504 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.377023 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.377155 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.377335 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.377372 (+ 37us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.377447 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:02:17.377619 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.373620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.0.0.1:52716)
0807 00:02:17.374006 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.374020 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.374160 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.374479 (+ 319us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.374482 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.374503 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:17.374626 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.374633 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.375495 (+ 862us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.375499 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.376342 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.376355 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.377155 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.377159 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.377259 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.377286 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.377353 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.377448 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":110,"threads_started":1}
I20260807 00:02:17.378922 9768 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260807 00:02:17.379511 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f64e620bad754807ad12eaad9ef7d37b. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:17.379680 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: This master's current role is: LEARNER
W20260807 00:02:17.467154 10251 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:17.467494 10251 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:17.467572 10251 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:17.471840 10251 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:17.471911 10251 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:17.471992 10251 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:17.477648 10251 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42613
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab
--principal=oryx/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:43509
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=46097
--tserver_master_addrs=127.30.204.126:35533,127.30.204.125:45769,127.30.204.124:46519
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10251
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:17.478827 10251 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:17.479954 10251 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:17.490455 10261 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:17.490461 10267 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:17.490497 10262 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:17.490554 10251 server_base.cc:1034] running on GCE node
I20260807 00:02:17.491278 10251 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:17.492142 10251 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:17.493336 10251 hybrid_clock.cc:648] HybridClock initialized: now 1786060937493312 us; error 38 us; skew 500 ppm
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:17.496549 10251 init.cc:377] Logged in from keytab as oryx/127.30.204.67@KRBTEST.COM (short username oryx)
I20260807 00:02:17.497535 10251 webserver.cc:460] Webserver started at http://127.30.204.67:46097/ using document root <none> and password file <none>
I20260807 00:02:17.498170 10251 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:17.498262 10251 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:17.502700 10251 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:17.505116 10276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.506548 10251 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:17.506697 10251 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "d96f92e64db74123b48da2b979cf8c85"
format_stamp: "Formatted at 2026-08-07 00:02:15 on dist-test-slave-pww0"
server_key: "0ab191587724ae0b423d1ad289c50b38"
server_key_iv: "108667508630ee3321a306c80bf934e8"
server_key_version: "encryptionkey@0"
I20260807 00:02:17.507279 10251 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:17.525244 10251 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:17.528852 10251 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:17.529101 10251 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:17.529814 10251 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:17.530812 10251 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:17.530865 10251 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.530925 10251 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:17.530956 10251 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:17.540362 10251 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:43509
I20260807 00:02:17.540391 10453 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:43509 every 8 connection(s)
I20260807 00:02:17.541600 10251 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:17.542346 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 10251
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.125@KRBTEST.COM
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
Aug 07 00:02:17 dist-test-slave-pww0 krb5kdc[8487](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060937, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.124@KRBTEST.COM
I20260807 00:02:17.558108 10478 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.544865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41652 (local address 127.30.204.126:35533)
0807 00:02:17.545523 (+ 658us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.545527 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.545542 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:17.545592 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.545595 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.545711 (+ 116us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.545822 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.546875 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.547807 (+ 932us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.549027 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.549394 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.549897 (+ 503us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.551423 (+ 1526us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.551428 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.551431 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.551503 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.553499 (+ 1996us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.555312 (+ 1813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.555315 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.555317 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.555425 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.556732 (+ 1307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.556735 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.556737 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.556974 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.557684 (+ 710us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.557772 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.557913 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":582,"thread_start_us":77,"threads_started":1}
I20260807 00:02:17.558974 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.545643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43090 (local address 127.30.204.125:45769)
0807 00:02:17.546032 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.546037 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.546828 (+ 791us) server_negotiation.cc:407] Connection header received
0807 00:02:17.546944 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.546947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.546998 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.547070 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.547970 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.548793 (+ 823us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.549887 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.550201 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.550308 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.551887 (+ 1579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.551891 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.551894 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.551922 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.556098 (+ 4176us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.556582 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.556597 (+ 15us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.556599 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.556695 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.557013 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.557017 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.557019 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.557352 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.557485 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.558651 (+ 1166us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.558793 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":322,"thread_start_us":97,"threads_started":1}
I20260807 00:02:17.561367 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.545654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52732 (local address 127.30.204.124:46519)
0807 00:02:17.547076 (+ 1422us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.547081 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.547095 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:17.547171 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.547174 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.547223 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.547337 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.548351 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.549126 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.550567 (+ 1441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.551179 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.551271 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.554688 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.554691 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.554694 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.554726 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.557600 (+ 2874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.558041 (+ 441us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.558044 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.558046 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.558155 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.559331 (+ 1176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.559334 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.559336 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.559597 (+ 261us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.559746 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.561059 (+ 1313us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.561195 (+ 136us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1344,"thread_start_us":84,"threads_started":1}
I20260807 00:02:17.564379 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.543971 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:50009 (local address 127.30.204.124:46519)
0807 00:02:17.544107 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.544113 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.546317 (+ 2204us) server_negotiation.cc:407] Connection header received
0807 00:02:17.550110 (+ 3793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.550113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.550167 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.550220 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.552320 (+ 2100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.553012 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.554638 (+ 1626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.554976 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.555057 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.559000 (+ 3943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.559004 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.559007 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.559033 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.561986 (+ 2953us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.562939 (+ 953us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.562943 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.562944 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.563029 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.563408 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.563412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.563413 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.563666 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.563746 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.564105 (+ 359us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.564226 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:02:17.564383 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.544736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:51903 (local address 127.30.204.125:45769)
0807 00:02:17.545078 (+ 342us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.545084 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.545577 (+ 493us) server_negotiation.cc:407] Connection header received
0807 00:02:17.547735 (+ 2158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.547738 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.547786 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.547836 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.550286 (+ 2450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.551011 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.552835 (+ 1824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.553181 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.553258 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.559444 (+ 6186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.559448 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.559451 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.559501 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.561543 (+ 2042us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.562277 (+ 734us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.562280 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.562282 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.562374 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.562934 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.562938 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.562940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.563200 (+ 260us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.563287 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.564098 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.564229 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259}
I20260807 00:02:17.564396 8604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.543826 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:46167 (local address 127.30.204.126:35533)
0807 00:02:17.544469 (+ 643us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.544474 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.546146 (+ 1672us) server_negotiation.cc:407] Connection header received
0807 00:02:17.547490 (+ 1344us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.547494 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.547542 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.547595 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.552419 (+ 4824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.553192 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.554821 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.555131 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.555214 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.558992 (+ 3778us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.558995 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.558998 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.559022 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.561252 (+ 2230us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.563112 (+ 1860us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.563116 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.563118 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.563209 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.563541 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.563545 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.563546 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.563752 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.563825 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.564152 (+ 327us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.564260 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":543}
I20260807 00:02:17.561334 8550 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41652:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:17.566028 8550 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:17.566222 10479 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.544316 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.30.204.67:51903)
0807 00:02:17.545422 (+ 1106us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.545483 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.547355 (+ 1872us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.549784 (+ 2429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.549787 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.549990 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.550123 (+ 133us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.550132 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.551204 (+ 1072us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.551208 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.552700 (+ 1492us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.552711 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.553469 (+ 758us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.553473 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.553896 (+ 423us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.555637 (+ 1741us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.555660 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.559293 (+ 3633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.561657 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.561665 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.561685 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.562159 (+ 474us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.562486 (+ 327us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.562489 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.562491 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.562823 (+ 332us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.563395 (+ 572us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.563402 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.563806 (+ 404us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.564543 (+ 737us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.565040 (+ 497us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":869,"mutex_wait_us":1069,"thread_start_us":704,"threads_started":1}
I20260807 00:02:17.566236 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.544462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.67:46167)
0807 00:02:17.546030 (+ 1568us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.546043 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.547288 (+ 1245us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.548351 (+ 1063us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.548362 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.548863 (+ 501us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.549583 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.549606 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.553840 (+ 4234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.553843 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.554693 (+ 850us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.554703 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.555385 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.555389 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.555516 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.555902 (+ 386us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.555916 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.558797 (+ 2881us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.562224 (+ 3427us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.562228 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.562230 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.562479 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.563312 (+ 833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.563315 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.563317 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.563409 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.563894 (+ 485us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.563896 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.563972 (+ 76us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.564574 (+ 602us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.565339 (+ 765us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1503,"mutex_wait_us":440,"spinlock_wait_cycles":28160,"thread_start_us":147,"threads_started":1}
I20260807 00:02:17.566741 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.545164 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.30.204.67:50009)
0807 00:02:17.546234 (+ 1070us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.546246 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.547076 (+ 830us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.551856 (+ 4780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.551859 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.552035 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:17.552182 (+ 147us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.552191 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.553171 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.553174 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.554493 (+ 1319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.554505 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.555232 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.555235 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.555336 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.556274 (+ 938us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:17.556289 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:17.558851 (+ 2562us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:17.562585 (+ 3734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.562587 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.562589 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.562827 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.563170 (+ 343us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:17.563172 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:17.563174 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:17.563308 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:17.563803 (+ 495us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:17.563806 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:17.563901 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.564916 (+ 1015us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.565108 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":996,"mutex_wait_us":31,"spinlock_wait_cycles":13696,"thread_start_us":92,"threads_started":1}
I20260807 00:02:17.567649 10458 heartbeater.cc:347] Connected to a master server at 127.30.204.124:46519
I20260807 00:02:17.567742 10460 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35533
I20260807 00:02:17.567691 10461 heartbeater.cc:347] Connected to a master server at 127.30.204.125:45769
I20260807 00:02:17.567945 10461 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.567942 10458 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.567942 10460 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:17.568630 10460 heartbeater.cc:511] Master 127.30.204.126:35533 requested a full tablet report, sending...
I20260807 00:02:17.568643 10461 heartbeater.cc:511] Master 127.30.204.125:45769 requested a full tablet report, sending...
I20260807 00:02:17.568630 10458 heartbeater.cc:511] Master 127.30.204.124:46519 requested a full tablet report, sending...
I20260807 00:02:17.569959 9756 ts_manager.cc:194] Registered new tserver with Master: d96f92e64db74123b48da2b979cf8c85 (127.30.204.67:43509)
I20260807 00:02:17.570076 9476 ts_manager.cc:194] Registered new tserver with Master: d96f92e64db74123b48da2b979cf8c85 (127.30.204.67:43509)
I20260807 00:02:17.570133 8549 ts_manager.cc:194] Re-registered known tserver with Master: d96f92e64db74123b48da2b979cf8c85 (127.30.204.67:43509)
I20260807 00:02:17.570616 9404 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: NON_VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: true } } }
I20260807 00:02:17.570741 9404 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:17.570892 8549 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:46167
I20260807 00:02:17.574961 10498 raft_consensus.cc:1050] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: attempting to promote NON_VOTER 4c1ec5ff506742d8a9ffaa32852aaca4 to VOTER
I20260807 00:02:17.575513 10498 consensus_queue.cc:237] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } }
I20260807 00:02:17.576793 9495 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Refusing update from remote peer f64e620bad754807ad12eaad9ef7d37b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 00:02:17.577656 9404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Connected to new peer: Peer: permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 00:02:17.578732 9768 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 LEARNER]: Refusing update from remote peer f64e620bad754807ad12eaad9ef7d37b: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260807 00:02:17.579164 9404 consensus_queue.cc:1035] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260807 00:02:17.579131 10496 raft_consensus.cc:2932] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.579604 9495 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.580394 9768 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.580852 10496 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.580955 10496 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:17.581096 10496 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.581215 10496 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:17.581192 10506 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.581316 10506 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:02:17.582319 9815 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.582440 9815 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:02:17.583120 9404 sys_catalog.cc:455] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "f64e620bad754807ad12eaad9ef7d37b" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:17.583233 9404 sys_catalog.cc:458] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:17.591529 10511 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.586338 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:43589 (local address 127.0.0.1:44156)
0807 00:02:17.586745 (+ 407us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.586758 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.586847 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.587565 (+ 718us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.587568 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.587588 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:17.587727 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.587734 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.589001 (+ 1267us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.589004 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.589900 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.589910 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.591071 (+ 1161us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.591075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.591178 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.591207 (+ 29us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.591258 (+ 51us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.591334 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"thread_start_us":97,"threads_started":1}
I20260807 00:02:17.592271 10252 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.585910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:34259 (local address 127.0.0.1:55982)
0807 00:02:17.586013 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.586028 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.586113 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.587954 (+ 1841us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.587957 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.587974 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:17.588062 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.588068 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.589634 (+ 1566us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.589637 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.590821 (+ 1184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.590832 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.591883 (+ 1051us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.591886 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.591973 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.591995 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.592057 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.592134 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":26,"mutex_wait_us":87}
I20260807 00:02:17.592322 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.586249 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55982 (local address 127.30.204.65:34259)
0807 00:02:17.587574 (+ 1325us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.587580 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.587600 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:02:17.587665 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.587669 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.587809 (+ 140us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.588218 (+ 409us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:17.588356 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.589506 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.590970 (+ 1464us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.591782 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.591880 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.592031 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.592091 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.592194 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1227,"thread_start_us":120,"threads_started":1}
I20260807 00:02:17.592511 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.586749 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44156 (local address 127.30.204.66:43589)
0807 00:02:17.587087 (+ 338us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.587098 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.587119 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:02:17.587188 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.587193 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.587377 (+ 184us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.587891 (+ 514us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:17.588046 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.588884 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.590033 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.590946 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.591971 (+ 1025us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.592111 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.592154 (+ 43us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.592237 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227,"thread_start_us":122,"threads_started":1}
I20260807 00:02:17.592563 10510 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.586487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:43509 (local address 127.0.0.1:55494)
0807 00:02:17.586857 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.586872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.586965 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.588095 (+ 1130us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.588098 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.588113 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:17.588193 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.588198 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.589606 (+ 1408us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.589610 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.590562 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.590572 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.592029 (+ 1457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.592033 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.592110 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.592129 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.592202 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.592262 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"mutex_wait_us":51,"thread_start_us":111,"threads_started":1}
I20260807 00:02:17.593001 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.587132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55494 (local address 127.30.204.67:43509)
0807 00:02:17.587429 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.587441 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.587487 (+ 46us) server_negotiation.cc:407] Connection header received
0807 00:02:17.587568 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.587574 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.587793 (+ 219us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.587953 (+ 160us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:17.588313 (+ 360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.589482 (+ 1169us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.590686 (+ 1204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.591562 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.591805 (+ 243us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.591980 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.592741 (+ 761us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.592858 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
I20260807 00:02:17.595665 10130 tablet_service.cc:1467] Processing CreateTablet for tablet bb469fd4265f47a9a86aa23658c59ae7 (DEFAULT_TABLE table=test-table [id=6f41fe72ff42460ba06e77930d39e648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:17.595654 10353 tablet_service.cc:1467] Processing CreateTablet for tablet bb469fd4265f47a9a86aa23658c59ae7 (DEFAULT_TABLE table=test-table [id=6f41fe72ff42460ba06e77930d39e648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:17.595652 9924 tablet_service.cc:1467] Processing CreateTablet for tablet bb469fd4265f47a9a86aa23658c59ae7 (DEFAULT_TABLE table=test-table [id=6f41fe72ff42460ba06e77930d39e648]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:17.596791 10130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb469fd4265f47a9a86aa23658c59ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:17.596799 9924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb469fd4265f47a9a86aa23658c59ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:17.596799 10353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bb469fd4265f47a9a86aa23658c59ae7. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:17.604648 10516 tablet_bootstrap.cc:492] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Bootstrap starting.
I20260807 00:02:17.605760 10517 tablet_bootstrap.cc:492] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Bootstrap starting.
I20260807 00:02:17.606534 10519 tablet_bootstrap.cc:492] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Bootstrap starting.
I20260807 00:02:17.607221 10516 tablet_bootstrap.cc:654] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:17.608045 10516 log.cc:824] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:17.608088 10517 tablet_bootstrap.cc:654] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:17.608891 10517 log.cc:824] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:17.608928 10519 tablet_bootstrap.cc:654] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:17.609730 10519 log.cc:824] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:17.610296 10516 tablet_bootstrap.cc:492] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: No bootstrap required, opened a new log
I20260807 00:02:17.610538 10516 ts_tablet_manager.cc:1397] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:02:17.611116 10517 tablet_bootstrap.cc:492] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: No bootstrap required, opened a new log
I20260807 00:02:17.611308 10517 ts_tablet_manager.cc:1397] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:02:17.612247 10519 tablet_bootstrap.cc:492] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: No bootstrap required, opened a new log
I20260807 00:02:17.612468 10519 ts_tablet_manager.cc:1397] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:02:17.614708 10516 raft_consensus.cc:348] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.614964 10516 raft_consensus.cc:374] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:17.615021 10516 raft_consensus.cc:729] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc326dcc80db47e4987e05da45070868, State: Initialized, Role: FOLLOWER
I20260807 00:02:17.615262 10517 raft_consensus.cc:348] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.615564 10517 raft_consensus.cc:374] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:17.615518 10516 consensus_queue.cc:260] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.615634 10517 raft_consensus.cc:729] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac8aa046d9641d09e66ab1e6c1e34a1, State: Initialized, Role: FOLLOWER
I20260807 00:02:17.616323 10236 heartbeater.cc:503] Master 127.30.204.126:35533 was elected leader, sending a full tablet report...
I20260807 00:02:17.616678 10516 ts_tablet_manager.cc:1428] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:02:17.617858 10517 consensus_queue.cc:260] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.618953 10517 ts_tablet_manager.cc:1428] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Time spent starting tablet: real 0.008s user 0.003s sys 0.004s
I20260807 00:02:17.619191 10519 raft_consensus.cc:348] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.619542 10519 raft_consensus.cc:374] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:17.619644 10519 raft_consensus.cc:729] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d96f92e64db74123b48da2b979cf8c85, State: Initialized, Role: FOLLOWER
I20260807 00:02:17.620146 10519 consensus_queue.cc:260] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.620370 10026 heartbeater.cc:503] Master 127.30.204.126:35533 was elected leader, sending a full tablet report...
I20260807 00:02:17.623368 10460 heartbeater.cc:503] Master 127.30.204.126:35533 was elected leader, sending a full tablet report...
I20260807 00:02:17.624718 10519 ts_tablet_manager.cc:1428] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Time spent starting tablet: real 0.012s user 0.007s sys 0.000s
I20260807 00:02:17.665490 10530 raft_consensus.cc:482] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:17.665720 10530 raft_consensus.cc:504] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.666958 10530 leader_election.cc:288] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259), dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
I20260807 00:02:17.672050 10507 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.667328 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:45677 (local address 127.30.204.65:34259)
0807 00:02:17.667451 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.667480 (+ 29us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.667611 (+ 131us) server_negotiation.cc:407] Connection header received
0807 00:02:17.667735 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.667737 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.667776 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.667838 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:17.668111 (+ 273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.668892 (+ 781us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.670672 (+ 1780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.671317 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.671451 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.671534 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.671831 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.671913 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:02:17.672068 10514 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.667334 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:43323 (local address 127.30.204.66:43589)
0807 00:02:17.667444 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.667448 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.667486 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:02:17.667659 (+ 173us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.667661 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.667705 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.667767 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:17.668097 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.668907 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.670672 (+ 1765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.671321 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.671451 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.671534 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.671831 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.671913 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:02:17.672086 10472 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.667265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:34259 (local address 127.30.204.67:45677)
0807 00:02:17.667532 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.667544 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.667637 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.667859 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.667862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.667877 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:17.667999 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.668006 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.669014 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.669018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.670511 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.670524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.671493 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.671497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.671659 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.671716 (+ 57us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.671793 (+ 77us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.671885 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":244}
I20260807 00:02:17.672086 10486 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.667250 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:43589 (local address 127.30.204.67:43323)
0807 00:02:17.667383 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:17.667399 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:17.667521 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:17.667790 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:17.667792 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:17.667809 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:17.667957 (+ 148us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.667965 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.669014 (+ 1049us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.669018 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.670511 (+ 1493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:17.670524 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.671493 (+ 969us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.671497 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:17.671658 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.671714 (+ 56us) client_negotiation.cc:769] Sending connection context
0807 00:02:17.671793 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 00:02:17.671885 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":46,"mutex_wait_us":112}
I20260807 00:02:17.672890 9955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb469fd4265f47a9a86aa23658c59ae7" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" is_pre_election: true
I20260807 00:02:17.673086 10164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb469fd4265f47a9a86aa23658c59ae7" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc326dcc80db47e4987e05da45070868" is_pre_election: true
I20260807 00:02:17.673269 9955 raft_consensus.cc:2454] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 0.
I20260807 00:02:17.673337 10164 raft_consensus.cc:2454] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 0.
I20260807 00:02:17.673806 10282 leader_election.cc:302] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2ac8aa046d9641d09e66ab1e6c1e34a1, d96f92e64db74123b48da2b979cf8c85; no voters:
I20260807 00:02:17.674117 10530 raft_consensus.cc:2781] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:17.674225 10530 raft_consensus.cc:482] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:17.674278 10530 raft_consensus.cc:3037] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:17.675518 10530 raft_consensus.cc:504] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.675884 10530 leader_election.cc:288] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 election: Requested vote from peers 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259), dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
I20260807 00:02:17.676326 9955 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb469fd4265f47a9a86aa23658c59ae7" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1"
I20260807 00:02:17.676326 10164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bb469fd4265f47a9a86aa23658c59ae7" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc326dcc80db47e4987e05da45070868"
I20260807 00:02:17.676486 9955 raft_consensus.cc:3037] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:17.676486 10164 raft_consensus.cc:3037] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:17.677874 9955 raft_consensus.cc:2454] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 1.
I20260807 00:02:17.677874 10164 raft_consensus.cc:2454] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 1.
I20260807 00:02:17.678256 10279 leader_election.cc:302] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d96f92e64db74123b48da2b979cf8c85, dc326dcc80db47e4987e05da45070868; no voters:
I20260807 00:02:17.678752 10530 raft_consensus.cc:2781] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:17.678977 10530 raft_consensus.cc:686] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 1 LEADER]: Becoming Leader. State: Replica: d96f92e64db74123b48da2b979cf8c85, State: Running, Role: LEADER
I20260807 00:02:17.679368 10530 consensus_queue.cc:237] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } }
I20260807 00:02:17.683017 8549 catalog_manager.cc:5495] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 reported cstate change: term changed from 0 to 1, leader changed from <none> to d96f92e64db74123b48da2b979cf8c85 (127.30.204.67). New cstate: current_term: 1 leader_uuid: "d96f92e64db74123b48da2b979cf8c85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } health_report { overall_health: HEALTHY } } }
I20260807 00:02:17.715857 10515 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.712135 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55502 (local address 127.30.204.67:43509)
0807 00:02:17.712265 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.712269 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.712282 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:02:17.712385 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.712387 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.712444 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.712533 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:17.712838 (+ 305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.713389 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.714272 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.714503 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.714601 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.714962 (+ 361us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:17.715425 (+ 463us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:17.715572 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.715650 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.715713 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:02:17.722393 9955 raft_consensus.cc:1261] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 1 FOLLOWER]: Refusing update from remote peer d96f92e64db74123b48da2b979cf8c85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:17.722391 10164 raft_consensus.cc:1261] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Refusing update from remote peer d96f92e64db74123b48da2b979cf8c85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:17.723169 10530 consensus_queue.cc:1035] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:17.723402 10532 consensus_queue.cc:1035] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:17.731335 10535 mvcc.cc:205] Tried to move back new op lower bound from 7315705600903032832 to 7315705600737669120. Current Snapshot: MvccSnapshot[applied={T|T < 7315705600903032832}]
I20260807 00:02:17.742004 8549 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:17.742195 8549 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41652:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:17.742681 10538 mvcc.cc:205] Tried to move back new op lower bound from 7315705600903032832 to 7315705600737669120. Current Snapshot: MvccSnapshot[applied={T|T < 7315705600903032832}]
I20260807 00:02:17.747401 8549 catalog_manager.cc:5782] T 00000000000000000000000000000000 P f64e620bad754807ad12eaad9ef7d37b: Sending DeleteTablet for 3 replicas of tablet bb469fd4265f47a9a86aa23658c59ae7
I20260807 00:02:17.748243 10130 tablet_service.cc:1514] Processing DeleteTablet for tablet bb469fd4265f47a9a86aa23658c59ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:17 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:44156
I20260807 00:02:17.748494 10128 tablet_service.cc:1514] Processing DeleteTablet for tablet bb469fd4265f47a9a86aa23658c59ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:17 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:44156
I20260807 00:02:17.748557 9924 tablet_service.cc:1514] Processing DeleteTablet for tablet bb469fd4265f47a9a86aa23658c59ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:17 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:55982
I20260807 00:02:17.748795 10353 tablet_service.cc:1514] Processing DeleteTablet for tablet bb469fd4265f47a9a86aa23658c59ae7 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:17 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:55494
I20260807 00:02:17.749120 10550 tablet_replica.cc:330] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: stopping tablet replica
I20260807 00:02:17.749570 10550 raft_consensus.cc:2229] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:17.750042 10550 raft_consensus.cc:2258] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:17.751649 10551 tablet_replica.cc:330] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: stopping tablet replica
I20260807 00:02:17.751940 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 8502
I20260807 00:02:17.752025 10551 raft_consensus.cc:2229] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:17.767287 10551 raft_consensus.cc:2258] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:17.767714 10550 ts_tablet_manager.cc:1905] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:17.771231 10550 ts_tablet_manager.cc:1918] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:17.771512 10550 log.cc:1196] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/bb469fd4265f47a9a86aa23658c59ae7
I20260807 00:02:17.771991 10550 ts_tablet_manager.cc:1939] T bb469fd4265f47a9a86aa23658c59ae7 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Deleting consensus metadata
I20260807 00:02:17.772024 10551 ts_tablet_manager.cc:1905] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:17.775172 10551 ts_tablet_manager.cc:1918] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:17.775308 10551 log.cc:1196] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/bb469fd4265f47a9a86aa23658c59ae7
I20260807 00:02:17.775687 10551 ts_tablet_manager.cc:1939] T bb469fd4265f47a9a86aa23658c59ae7 P d96f92e64db74123b48da2b979cf8c85: Deleting consensus metadata
I20260807 00:02:17.775997 10549 tablet_replica.cc:330] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: stopping tablet replica
I20260807 00:02:17.776636 10549 raft_consensus.cc:2229] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:17.778589 10549 raft_consensus.cc:2258] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 00:02:17.780603 10279 connection.cc:535] server connection from 127.0.0.1:55494 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:02:17.780721 10549 ts_tablet_manager.cc:1905] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 00:02:17.781344 9851 connection.cc:535] server connection from 127.0.0.1:55982 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:02:17.783799 10549 ts_tablet_manager.cc:1918] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:17.783912 10549 log.cc:1196] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/bb469fd4265f47a9a86aa23658c59ae7
I20260807 00:02:17.784291 10549 ts_tablet_manager.cc:1939] T bb469fd4265f47a9a86aa23658c59ae7 P dc326dcc80db47e4987e05da45070868: Deleting consensus metadata
W20260807 00:02:17.785780 10062 connection.cc:441] server connection from 127.0.0.1:44156 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44156 (request call id 1) could send its response
W20260807 00:02:17.786564 10062 connection.cc:441] server connection from 127.0.0.1:44156 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:44156 (request call id 2) could send its response
I20260807 00:02:17.788161 9662 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.750443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43092 (local address 127.30.204.125:45769)
0807 00:02:17.750599 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.750605 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.776246 (+ 25641us) server_negotiation.cc:407] Connection header received
0807 00:02:17.777029 (+ 783us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.777034 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.777094 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.777163 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.779017 (+ 1854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.779902 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.781318 (+ 1416us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.781672 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.781748 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.783873 (+ 2125us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.783878 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.783880 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.783914 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.785821 (+ 1907us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.786408 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.786412 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.786414 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.786510 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.787083 (+ 573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.787087 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.787089 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.787343 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.787452 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.787858 (+ 406us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.787982 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:02:17.796535 10489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.786064 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43100 (local address 127.30.204.125:45769)
0807 00:02:17.786192 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.786196 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.787003 (+ 807us) server_negotiation.cc:407] Connection header received
0807 00:02:17.787199 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.787203 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.787261 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.787312 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.788145 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.788931 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.790108 (+ 1177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.790466 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.790547 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.792282 (+ 1735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.792286 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.792289 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.792318 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.794303 (+ 1985us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.794961 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.794964 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.794966 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.795059 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.795353 (+ 294us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.795357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.795359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.795739 (+ 380us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.795849 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.796203 (+ 354us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.796355 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:02:17.797878 10487 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.786845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52744 (local address 127.30.204.124:46519)
0807 00:02:17.787012 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.787017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.787724 (+ 707us) server_negotiation.cc:407] Connection header received
0807 00:02:17.787905 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.787908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.787963 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.788017 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.789171 (+ 1154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.789974 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.791135 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.791625 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.791747 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.793360 (+ 1613us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.793364 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.793367 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.793400 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.794973 (+ 1573us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.795447 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.795451 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.795452 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.795728 (+ 276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.796081 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.796085 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.796086 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.796328 (+ 242us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.797477 (+ 1149us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.797570 (+ 93us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.797707 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":23,"server-negotiator.queue_time_us":68}
I20260807 00:02:17.800064 10032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:17.750700 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52740 (local address 127.30.204.124:46519)
0807 00:02:17.750832 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:17.750836 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:17.789900 (+ 39064us) server_negotiation.cc:407] Connection header received
0807 00:02:17.790055 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:17.790058 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:17.790118 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:17.790173 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:17.791373 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.792513 (+ 1140us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.793632 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:17.793968 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:17.794039 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:17.796255 (+ 2216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:17.796260 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:17.796263 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:17.796291 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:17.798266 (+ 1975us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.798745 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.798748 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.798750 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.798859 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:17.799159 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:17.799162 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:17.799164 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:17.799389 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:17.799550 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:17.799755 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 00:02:17.799874 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":53}
I20260807 00:02:18.735852 10562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:18.735088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.65:60249)
0807 00:02:18.735601 (+ 513us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:18.735732 (+ 131us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":293,"thread_start_us":97,"threads_started":1}
W20260807 00:02:18.736374 10026 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:35533 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:18.744673 10563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:18.744203 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.67:44081)
0807 00:02:18.744481 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:18.744569 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":88,"threads_started":1}
W20260807 00:02:18.745000 10460 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:35533 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:18.748944 10564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:18.748489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.66:51681)
0807 00:02:18.748756 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:18.748848 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":186,"thread_start_us":75,"threads_started":1}
W20260807 00:02:18.749249 10236 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:35533 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:19.247896 10566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f64e620bad754807ad12eaad9ef7d37b)
I20260807 00:02:19.248116 10566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } }
I20260807 00:02:19.249184 10566 leader_election.cc:288] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f64e620bad754807ad12eaad9ef7d37b (127.30.204.126:35533), 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124:46519)
I20260807 00:02:19.249853 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.249368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42106)
0807 00:02:19.249640 (+ 272us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.249739 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":180,"thread_start_us":67,"threads_started":1}
W20260807 00:02:19.250049 9456 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:19.250648 10574 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader f64e620bad754807ad12eaad9ef7d37b)
I20260807 00:02:19.250830 10574 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } }
I20260807 00:02:19.251034 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.250771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42108)
0807 00:02:19.250899 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.250959 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 00:02:19.251179 9456 leader_election.cc:334] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f64e620bad754807ad12eaad9ef7d37b (127.30.204.126:35533): Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:19.251991 10574 leader_election.cc:288] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers f64e620bad754807ad12eaad9ef7d37b (127.30.204.126:35533), 20b55adeb95b4d849c8d2cab53d50fb7 (127.30.204.125:45769)
I20260807 00:02:19.252530 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.252075 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42114)
0807 00:02:19.252363 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.252435 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":78,"threads_started":1}
W20260807 00:02:19.252717 9733 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:19.253603 10578 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.253374 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42120)
0807 00:02:19.253480 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.253536 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 00:02:19.253772 9733 leader_election.cc:334] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer f64e620bad754807ad12eaad9ef7d37b (127.30.204.126:35533): Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:19.253981 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.249368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:46519 (local address 127.0.0.1:52752)
0807 00:02:19.249645 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.249658 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:19.249777 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:19.250135 (+ 358us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:19.250139 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:19.250163 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:19.250291 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.250300 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.251374 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.251378 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.252718 (+ 1340us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.252730 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.253562 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.253566 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.253668 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.253698 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 00:02:19.253771 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:02:19.253843 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":29,"thread_start_us":58,"threads_started":1}
I20260807 00:02:19.254014 10572 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.249608 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52752 (local address 127.30.204.124:46519)
0807 00:02:19.249875 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:19.249879 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:19.249893 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:19.249952 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:19.249955 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:19.250004 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:19.250092 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:19.250424 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.251216 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.252859 (+ 1643us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.253451 (+ 592us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.253572 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.253626 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:19.253807 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 00:02:19.253893 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":103,"threads_started":1}
I20260807 00:02:19.254495 9768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" is_pre_election: true
I20260807 00:02:19.254738 9768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 20b55adeb95b4d849c8d2cab53d50fb7 in term 1.
I20260807 00:02:19.255164 9455 leader_election.cc:302] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20b55adeb95b4d849c8d2cab53d50fb7, 4c1ec5ff506742d8a9ffaa32852aaca4; no voters: f64e620bad754807ad12eaad9ef7d37b
I20260807 00:02:19.255493 10566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260807 00:02:19.255564 10566 raft_consensus.cc:482] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Starting leader election (detected failure of leader f64e620bad754807ad12eaad9ef7d37b)
I20260807 00:02:19.255612 10566 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:02:19.256476 10579 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.252281 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43102 (local address 127.30.204.125:45769)
0807 00:02:19.252563 (+ 282us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:19.252572 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:19.252764 (+ 192us) server_negotiation.cc:407] Connection header received
0807 00:02:19.252805 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:19.252808 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:19.252854 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:19.252919 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:19.253220 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.254017 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.255410 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.255971 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.256072 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.256112 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:19.256313 (+ 201us) server_negotiation.cc:299] Negotiation successful
0807 00:02:19.256372 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":96,"threads_started":1}
I20260807 00:02:19.256531 10577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.252130 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:45769 (local address 127.0.0.1:43102)
0807 00:02:19.252551 (+ 421us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.252564 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:19.252685 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:19.253020 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:19.253022 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:19.253039 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:19.253115 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.253119 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.254158 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.254161 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.255288 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.255299 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.256101 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.256104 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.256196 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.256223 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:02:19.256301 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 00:02:19.256379 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":356,"thread_start_us":59,"threads_started":1}
I20260807 00:02:19.257036 9495 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" is_pre_election: true
I20260807 00:02:19.257040 10566 raft_consensus.cc:504] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } }
I20260807 00:02:19.257308 9495 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 2 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate 4c1ec5ff506742d8a9ffaa32852aaca4 in current term 2: Already voted for candidate 20b55adeb95b4d849c8d2cab53d50fb7 in this term.
I20260807 00:02:19.257508 10566 leader_election.cc:288] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [CANDIDATE]: Term 2 election: Requested vote from peers f64e620bad754807ad12eaad9ef7d37b (127.30.204.126:35533), 4c1ec5ff506742d8a9ffaa32852aaca4 (127.30.204.124:46519)
I20260807 00:02:19.257911 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.257625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42134)
0807 00:02:19.257792 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.257848 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 00:02:19.257870 9768 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4"
I20260807 00:02:19.258001 9768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:02:19.258054 9734 leader_election.cc:302] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 4c1ec5ff506742d8a9ffaa32852aaca4; no voters: 20b55adeb95b4d849c8d2cab53d50fb7, f64e620bad754807ad12eaad9ef7d37b
I20260807 00:02:19.259084 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.258892 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42142)
0807 00:02:19.258978 (+ 86us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.259025 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 00:02:19.259197 9456 leader_election.cc:334] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer f64e620bad754807ad12eaad9ef7d37b (127.30.204.126:35533): Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
I20260807 00:02:19.259378 9768 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 2 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 20b55adeb95b4d849c8d2cab53d50fb7 in term 2.
I20260807 00:02:19.259511 10574 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 2 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:02:19.259724 9455 leader_election.cc:302] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 20b55adeb95b4d849c8d2cab53d50fb7, 4c1ec5ff506742d8a9ffaa32852aaca4; no voters: f64e620bad754807ad12eaad9ef7d37b
I20260807 00:02:19.259919 10566 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:02:19.260154 10566 raft_consensus.cc:686] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [term 2 LEADER]: Becoming Leader. State: Replica: 20b55adeb95b4d849c8d2cab53d50fb7, State: Running, Role: LEADER
I20260807 00:02:19.260426 10566 consensus_queue.cc:237] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 2, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } }
I20260807 00:02:19.262053 10581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20b55adeb95b4d849c8d2cab53d50fb7. Latest consensus state: current_term: 2 leader_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:19.262216 10581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:19.262555 10584 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:19.265200 10584 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:19.265666 10584 catalog_manager.cc:1259] Loaded cluster ID: 9e4d3f548fa74b3eabbb81737eb7d398
I20260807 00:02:19.265714 10584 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:19.266254 10584 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:19.266688 10584 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Loaded TSK: 0
I20260807 00:02:19.267395 10584 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:02:19.320991 9476 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43100:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:19.324280 9476 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:19.327764 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.327425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42166)
0807 00:02:19.327602 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.327676 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91}
I20260807 00:02:19.327811 9768 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [term 2 FOLLOWER]: Refusing update from remote peer 20b55adeb95b4d849c8d2cab53d50fb7: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 2 index: 14. (index mismatch)
I20260807 00:02:19.328382 10581 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260807 00:02:19.328516 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.328294 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.0.0.1:42168)
0807 00:02:19.328400 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.328453 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 00:02:19.328768 9456 consensus_peers.cc:487] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 -> Peer f64e620bad754807ad12eaad9ef7d37b (127.30.204.126:35533): Couldn't send request to peer f64e620bad754807ad12eaad9ef7d37b. Status: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260807 00:02:19.330169 10574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 20b55adeb95b4d849c8d2cab53d50fb7. Latest consensus state: current_term: 2 leader_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:19.330354 10574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:02:19.331600 10581 sys_catalog.cc:455] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 2 leader_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:19.331746 10581 sys_catalog.cc:458] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:19.332594 10574 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 2 leader_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "f64e620bad754807ad12eaad9ef7d37b" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35533 } } peers { permanent_uuid: "20b55adeb95b4d849c8d2cab53d50fb7" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 45769 } attrs { promote: false } } peers { permanent_uuid: "4c1ec5ff506742d8a9ffaa32852aaca4" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 46519 } attrs { promote: false } } }
I20260807 00:02:19.332728 10574 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4c1ec5ff506742d8a9ffaa32852aaca4 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:02:19.343504 10569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.338684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:34259 (local address 127.0.0.1:55988)
0807 00:02:19.338938 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.338953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:19.339044 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:19.339955 (+ 911us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:19.339958 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:19.339984 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:19.340070 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.340076 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.341121 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.341125 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.342192 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.342200 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.343064 (+ 864us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.343068 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.343179 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.343203 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:02:19.343252 (+ 49us) client_negotiation.cc:240] Negotiation successful
0807 00:02:19.343331 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"mutex_wait_us":1,"spinlock_wait_cycles":10368}
I20260807 00:02:19.343695 10597 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.339360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44170 (local address 127.30.204.66:43589)
0807 00:02:19.339627 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:19.339631 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:19.339642 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:02:19.339687 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:19.339690 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:19.339736 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:19.339815 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:19.340190 (+ 375us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.340843 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.342039 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.342483 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.342603 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.342691 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:19.343410 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 00:02:19.343565 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":195,"thread_start_us":105,"threads_started":1}
I20260807 00:02:19.343685 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.338597 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55504 (local address 127.30.204.67:43509)
0807 00:02:19.338865 (+ 268us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:19.338871 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:19.339063 (+ 192us) server_negotiation.cc:407] Connection header received
0807 00:02:19.339152 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:19.339154 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:19.339213 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:19.339688 (+ 475us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:19.339810 (+ 122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.340535 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.342047 (+ 1512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.342652 (+ 605us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.342794 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.342863 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:19.343409 (+ 546us) server_negotiation.cc:299] Negotiation successful
0807 00:02:19.343542 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182,"thread_start_us":98,"threads_started":1}
I20260807 00:02:19.343348 10595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.339010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:43589 (local address 127.0.0.1:44170)
0807 00:02:19.339320 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.339333 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:19.339421 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:19.339936 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:19.339939 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:19.339957 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:19.340051 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.340056 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.340951 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.340954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.341848 (+ 894us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.341860 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.342974 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.342978 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.343063 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.343086 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:02:19.343153 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:02:19.343215 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":247,"spinlock_wait_cycles":2560,"thread_start_us":152,"threads_started":1}
I20260807 00:02:19.343346 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.338461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:43509 (local address 127.0.0.1:55504)
0807 00:02:19.338938 (+ 477us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.338953 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:19.339044 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:19.339340 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:19.339343 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:19.339362 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:19.339534 (+ 172us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.339544 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.340676 (+ 1132us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.340679 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.341883 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:19.341893 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.342829 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.342833 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:19.342930 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.342962 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 00:02:19.343057 (+ 95us) client_negotiation.cc:240] Negotiation successful
0807 00:02:19.343157 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":402,"spinlock_wait_cycles":13568}
I20260807 00:02:19.344566 10596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.339207 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55988 (local address 127.30.204.65:34259)
0807 00:02:19.339702 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:19.339706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:19.339721 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:19.339770 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:19.339773 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:19.339818 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:19.340188 (+ 370us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:19.340300 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.340994 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.342337 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.342944 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.343896 (+ 952us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.344333 (+ 437us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:19.344377 (+ 44us) server_negotiation.cc:299] Negotiation successful
0807 00:02:19.344458 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":70,"threads_started":1}
I20260807 00:02:19.344583 10353 tablet_service.cc:1467] Processing CreateTablet for tablet 24bb82b929db4bb8a42dd55550c65bd0 (DEFAULT_TABLE table=test-table [id=6feeaf429bc446718fd372082b6fe048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:19.344583 10130 tablet_service.cc:1467] Processing CreateTablet for tablet 24bb82b929db4bb8a42dd55550c65bd0 (DEFAULT_TABLE table=test-table [id=6feeaf429bc446718fd372082b6fe048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:19.344916 10353 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24bb82b929db4bb8a42dd55550c65bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:19.344916 10130 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24bb82b929db4bb8a42dd55550c65bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:19.345046 9924 tablet_service.cc:1467] Processing CreateTablet for tablet 24bb82b929db4bb8a42dd55550c65bd0 (DEFAULT_TABLE table=test-table [id=6feeaf429bc446718fd372082b6fe048]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:19.345345 9924 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 24bb82b929db4bb8a42dd55550c65bd0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:19.347990 10598 tablet_bootstrap.cc:492] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Bootstrap starting.
I20260807 00:02:19.348191 10599 tablet_bootstrap.cc:492] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Bootstrap starting.
I20260807 00:02:19.349189 10599 tablet_bootstrap.cc:654] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:19.349189 10598 tablet_bootstrap.cc:654] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:19.349933 10600 tablet_bootstrap.cc:492] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Bootstrap starting.
I20260807 00:02:19.350732 10598 tablet_bootstrap.cc:492] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: No bootstrap required, opened a new log
I20260807 00:02:19.350839 10598 ts_tablet_manager.cc:1397] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:19.350910 10600 tablet_bootstrap.cc:654] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:19.351701 10599 tablet_bootstrap.cc:492] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1: No bootstrap required, opened a new log
I20260807 00:02:19.351795 10599 ts_tablet_manager.cc:1397] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 00:02:19.352344 10600 tablet_bootstrap.cc:492] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: No bootstrap required, opened a new log
I20260807 00:02:19.352321 10599 raft_consensus.cc:348] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.352447 10600 ts_tablet_manager.cc:1397] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 00:02:19.352460 10599 raft_consensus.cc:374] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:19.352494 10599 raft_consensus.cc:729] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2ac8aa046d9641d09e66ab1e6c1e34a1, State: Initialized, Role: FOLLOWER
I20260807 00:02:19.352654 10599 consensus_queue.cc:260] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.352893 10600 raft_consensus.cc:348] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.353011 10600 raft_consensus.cc:374] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:19.353037 10599 ts_tablet_manager.cc:1428] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:19.353048 10600 raft_consensus.cc:729] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d96f92e64db74123b48da2b979cf8c85, State: Initialized, Role: FOLLOWER
I20260807 00:02:19.353221 10600 consensus_queue.cc:260] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.353572 10600 ts_tablet_manager.cc:1428] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Time spent starting tablet: real 0.001s user 0.000s sys 0.002s
I20260807 00:02:19.353593 10598 raft_consensus.cc:348] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.353713 10598 raft_consensus.cc:374] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:19.353744 10598 raft_consensus.cc:729] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dc326dcc80db47e4987e05da45070868, State: Initialized, Role: FOLLOWER
I20260807 00:02:19.353894 10598 consensus_queue.cc:260] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.354231 10598 ts_tablet_manager.cc:1428] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Time spent starting tablet: real 0.001s user 0.002s sys 0.000s
I20260807 00:02:19.356701 10027 heartbeater.cc:503] Master 127.30.204.125:45769 was elected leader, sending a full tablet report...
I20260807 00:02:19.357372 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.356260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.66:52195)
0807 00:02:19.357181 (+ 921us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.357243 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":854,"thread_start_us":100,"threads_started":1}
I20260807 00:02:19.357654 10610 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.353667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.65:44459)
0807 00:02:19.356938 (+ 3271us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.357528 (+ 590us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3189,"thread_start_us":117,"threads_started":1}
I20260807 00:02:19.358608 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.358112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.65:53241)
0807 00:02:19.358439 (+ 327us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.358504 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":146,"threads_started":1}
I20260807 00:02:19.359072 10461 heartbeater.cc:503] Master 127.30.204.125:45769 was elected leader, sending a full tablet report...
I20260807 00:02:19.359113 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.358628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.66:52761)
0807 00:02:19.358989 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.359043 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292}
I20260807 00:02:19.359375 10239 heartbeater.cc:503] Master 127.30.204.125:45769 was elected leader, sending a full tablet report...
I20260807 00:02:19.359799 10613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.355286 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.67:37709)
0807 00:02:19.359633 (+ 4347us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.359698 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4275,"thread_start_us":108,"threads_started":1}
I20260807 00:02:19.360625 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.360237 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.67:36747)
0807 00:02:19.360508 (+ 271us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.360556 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":121,"threads_started":1}
W20260807 00:02:19.536640 10028 tablet.cc:2369] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:02:19.546272 10462 tablet.cc:2369] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:02:19.560416 10609 raft_consensus.cc:482] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:19.560659 10609 raft_consensus.cc:504] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.561177 10609 leader_election.cc:288] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259), dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
I20260807 00:02:19.561755 10164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24bb82b929db4bb8a42dd55550c65bd0" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc326dcc80db47e4987e05da45070868" is_pre_election: true
I20260807 00:02:19.561827 9954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24bb82b929db4bb8a42dd55550c65bd0" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" is_pre_election: true
I20260807 00:02:19.561949 10164 raft_consensus.cc:2454] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 0.
I20260807 00:02:19.561982 9954 raft_consensus.cc:2454] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 0.
I20260807 00:02:19.562356 10279 leader_election.cc:302] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d96f92e64db74123b48da2b979cf8c85, dc326dcc80db47e4987e05da45070868; no voters:
I20260807 00:02:19.562577 10609 raft_consensus.cc:2781] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:19.562662 10609 raft_consensus.cc:482] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:19.562702 10609 raft_consensus.cc:3037] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 0 FOLLOWER]: Advancing to term 1
W20260807 00:02:19.563282 10241 tablet.cc:2369] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:02:19.563982 10609 raft_consensus.cc:504] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.564391 10609 leader_election.cc:288] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 election: Requested vote from peers 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259), dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589)
I20260807 00:02:19.564855 10164 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24bb82b929db4bb8a42dd55550c65bd0" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "dc326dcc80db47e4987e05da45070868"
I20260807 00:02:19.565013 10164 raft_consensus.cc:3037] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:19.565004 9954 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "24bb82b929db4bb8a42dd55550c65bd0" candidate_uuid: "d96f92e64db74123b48da2b979cf8c85" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1"
I20260807 00:02:19.565137 9954 raft_consensus.cc:3037] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:19.566301 10164 raft_consensus.cc:2454] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 1.
I20260807 00:02:19.566301 9954 raft_consensus.cc:2454] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d96f92e64db74123b48da2b979cf8c85 in term 1.
I20260807 00:02:19.566665 10279 leader_election.cc:302] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: d96f92e64db74123b48da2b979cf8c85, dc326dcc80db47e4987e05da45070868; no voters:
I20260807 00:02:19.566867 10609 raft_consensus.cc:2781] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:19.567154 10609 raft_consensus.cc:686] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 1 LEADER]: Becoming Leader. State: Replica: d96f92e64db74123b48da2b979cf8c85, State: Running, Role: LEADER
I20260807 00:02:19.567305 10609 consensus_queue.cc:237] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } }
I20260807 00:02:19.568110 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.567807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.67:53721)
0807 00:02:19.567939 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.568012 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260807 00:02:19.569846 9475 catalog_manager.cc:5495] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 reported cstate change: term changed from 0 to 1, leader changed from <none> to d96f92e64db74123b48da2b979cf8c85 (127.30.204.67). New cstate: current_term: 1 leader_uuid: "d96f92e64db74123b48da2b979cf8c85" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d96f92e64db74123b48da2b979cf8c85" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43509 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 } health_report { overall_health: UNKNOWN } } }
I20260807 00:02:19.586304 10593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.582847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55508 (local address 127.30.204.67:43509)
0807 00:02:19.583010 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:19.583014 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:19.583028 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:19.583089 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:19.583091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:19.583156 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:19.583227 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:19.583574 (+ 347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.584168 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.585140 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:19.585375 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:19.585460 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:19.585744 (+ 284us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:19.585819 (+ 75us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:19.585868 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:19.585989 (+ 121us) server_negotiation.cc:299] Negotiation successful
0807 00:02:19.586077 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260807 00:02:19.588826 10164 raft_consensus.cc:1261] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Refusing update from remote peer d96f92e64db74123b48da2b979cf8c85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:19.588871 9954 raft_consensus.cc:1261] T 24bb82b929db4bb8a42dd55550c65bd0 P 2ac8aa046d9641d09e66ab1e6c1e34a1 [term 1 FOLLOWER]: Refusing update from remote peer d96f92e64db74123b48da2b979cf8c85: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:19.589313 10609 consensus_queue.cc:1035] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "dc326dcc80db47e4987e05da45070868" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 43589 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:19.589465 10617 consensus_queue.cc:1035] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2ac8aa046d9641d09e66ab1e6c1e34a1" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34259 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:19.591396 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.591035 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.65:59121)
0807 00:02:19.591222 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.591305 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111}
I20260807 00:02:19.591513 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.591219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.66:36697)
0807 00:02:19.591341 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.591406 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260807 00:02:19.596035 10616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.595556 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.67:49325)
0807 00:02:19.595882 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.595948 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248}
I20260807 00:02:19.596621 10614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.596345 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.66:42815)
0807 00:02:19.596492 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.596546 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 00:02:19.597175 10627 mvcc.cc:205] Tried to move back new op lower bound from 7315705608551387136 to 7315705608469553152. Current Snapshot: MvccSnapshot[applied={T|T < 7315705608551387136}]
I20260807 00:02:19.598557 10615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.598274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35533 (local address 127.30.204.65:43273)
0807 00:02:19.598424 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.598481 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:35533: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 00:02:19.600253 9476 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:19.600417 9476 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43100:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:19.604256 9476 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 20b55adeb95b4d849c8d2cab53d50fb7: Sending DeleteTablet for 3 replicas of tablet 24bb82b929db4bb8a42dd55550c65bd0
I20260807 00:02:19.604931 10353 tablet_service.cc:1514] Processing DeleteTablet for tablet 24bb82b929db4bb8a42dd55550c65bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:19 UTC) from {username='oryx', principal='oryx/127.30.204.125@KRBTEST.COM'} at 127.0.0.1:55504
I20260807 00:02:19.605014 9924 tablet_service.cc:1514] Processing DeleteTablet for tablet 24bb82b929db4bb8a42dd55550c65bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:19 UTC) from {username='oryx', principal='oryx/127.30.204.125@KRBTEST.COM'} at 127.0.0.1:55988
I20260807 00:02:19.605163 10130 tablet_service.cc:1514] Processing DeleteTablet for tablet 24bb82b929db4bb8a42dd55550c65bd0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:19 UTC) from {username='oryx', principal='oryx/127.30.204.125@KRBTEST.COM'} at 127.0.0.1:44170
I20260807 00:02:19.605511 10640 tablet_replica.cc:330] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: stopping tablet replica
I20260807 00:02:19.605521 10646 tablet_replica.cc:330] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: stopping tablet replica
I20260807 00:02:19.605655 10640 raft_consensus.cc:2229] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:19.605655 10646 raft_consensus.cc:2229] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:19.605791 10646 raft_consensus.cc:2258] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:19.605870 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 9696
I20260807 00:02:19.605871 10640 raft_consensus.cc:2258] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:19.611632 10640 ts_tablet_manager.cc:1905] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:19.614126 10640 ts_tablet_manager.cc:1918] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:19.614225 10640 log.cc:1196] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/24bb82b929db4bb8a42dd55550c65bd0
I20260807 00:02:19.614529 10640 ts_tablet_manager.cc:1939] T 24bb82b929db4bb8a42dd55550c65bd0 P d96f92e64db74123b48da2b979cf8c85: Deleting consensus metadata
I20260807 00:02:19.616034 10646 ts_tablet_manager.cc:1905] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:19.617496 9457 catalog_manager.cc:4841] TS d96f92e64db74123b48da2b979cf8c85 (127.30.204.67:43509): tablet 24bb82b929db4bb8a42dd55550c65bd0 (table test-table [id=6feeaf429bc446718fd372082b6fe048]) successfully deleted
I20260807 00:02:19.618767 10646 ts_tablet_manager.cc:1918] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:19.618852 10646 log.cc:1196] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/24bb82b929db4bb8a42dd55550c65bd0
I20260807 00:02:19.619074 10646 ts_tablet_manager.cc:1939] T 24bb82b929db4bb8a42dd55550c65bd0 P dc326dcc80db47e4987e05da45070868: Deleting consensus metadata
I20260807 00:02:19.619920 9455 catalog_manager.cc:4841] TS dc326dcc80db47e4987e05da45070868 (127.30.204.66:43589): tablet 24bb82b929db4bb8a42dd55550c65bd0 (table test-table [id=6feeaf429bc446718fd372082b6fe048]) successfully deleted
I20260807 00:02:19.623479 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 10036
I20260807 00:02:19.626243 10568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:19.623689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:34259 (local address 127.0.0.1:56002)
0807 00:02:19.626064 (+ 2375us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:19.626139 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:34259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":2269}
W20260807 00:02:19.627689 9457 catalog_manager.cc:4584] TS 2ac8aa046d9641d09e66ab1e6c1e34a1 (127.30.204.65:34259): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 24bb82b929db4bb8a42dd55550c65bd0: Network error: Client connection negotiation failed: client connection to 127.30.204.65:34259: connect: Connection refused (error 111)
I20260807 00:02:19.636359 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 10251
I20260807 00:02:19.657047 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 9259
I20260807 00:02:19.673305 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 9420
2026-08-07T00:02:19Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (6040 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:19 dist-test-slave-pww0 krb5kdc[10650](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:19 dist-test-slave-pww0 krb5kdc[10650](info): set up 2 sockets
Aug 07 00:02:19 dist-test-slave-pww0 krb5kdc[10650](info): commencing operation
krb5kdc: starting...
W20260807 00:02:20.850657 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:20 dist-test-slave-pww0 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:20Z Disabled control of system clock
WARNING: no policy specified for oryx/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab.
Entry for principal oryx/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab.
I20260807 00:02:20.990619 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34527
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab
--principal=oryx/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:34527
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:21.119055 10671 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:21.119482 10671 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:21.119545 10671 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:21.125485 10671 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:21.125598 10671 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260807 00:02:21.125630 10671 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:21.125656 10671 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:21.125689 10671 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:21.125718 10671 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:21.134121 10671 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:34527
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.126.keytab
--principal=oryx/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10671
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:21.135874 10671 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:21.137348 10671 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:21.147850 10682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:21.147830 10683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:21.148177 10671 server_base.cc:1034] running on GCE node
W20260807 00:02:21.148375 10685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:21.149024 10671 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:21.150309 10671 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:21.151531 10671 hybrid_clock.cc:648] HybridClock initialized: now 1786060941151504 us; error 69 us; skew 500 ppm
Aug 07 00:02:21 dist-test-slave-pww0 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:21.160862 10671 init.cc:377] Logged in from keytab as oryx/127.30.204.126@KRBTEST.COM (short username oryx)
I20260807 00:02:21.161965 10671 webserver.cc:460] Webserver started at http://127.30.204.126:45111/ using document root <none> and password file <none>
I20260807 00:02:21.162590 10671 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:21.162676 10671 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:21.162911 10671 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:21.165212 10671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "27484d8abb864e2da97fd6353f869cb0"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "8005d10f3ed17bd064ebfb09c3be7c55"
server_key_iv: "a13e5d24a6110b20dfd0c365e32e9ece"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.165803 10671 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "27484d8abb864e2da97fd6353f869cb0"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "8005d10f3ed17bd064ebfb09c3be7c55"
server_key_iv: "a13e5d24a6110b20dfd0c365e32e9ece"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.170013 10671 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:21.172892 10698 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:21.174396 10671 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:21.174582 10671 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "27484d8abb864e2da97fd6353f869cb0"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "8005d10f3ed17bd064ebfb09c3be7c55"
server_key_iv: "a13e5d24a6110b20dfd0c365e32e9ece"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.174712 10671 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:21.196607 10671 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:21.200311 10671 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:21.200524 10671 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:21.208976 10671 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:34527
I20260807 00:02:21.208997 10782 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:34527 every 8 connection(s)
I20260807 00:02:21.210081 10671 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:21.213339 10784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:21.219601 10784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Bootstrap starting.
I20260807 00:02:21.219965 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 10671
I20260807 00:02:21.220079 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:21.220376 31537 external_mini_cluster.cc:1428] Setting key aa2ffb2514fb51fa4ec1d123e994567f
I20260807 00:02:21.222460 10784 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:21.223301 10784 log.cc:824] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:21.225513 10784 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: No bootstrap required, opened a new log
I20260807 00:02:21.229444 10784 raft_consensus.cc:348] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27484d8abb864e2da97fd6353f869cb0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34527 } }
Aug 07 00:02:21 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060940, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:21.229763 10784 raft_consensus.cc:374] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:21.229848 10784 raft_consensus.cc:729] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 27484d8abb864e2da97fd6353f869cb0, State: Initialized, Role: FOLLOWER
I20260807 00:02:21.230402 10784 consensus_queue.cc:260] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27484d8abb864e2da97fd6353f869cb0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34527 } }
I20260807 00:02:21.230603 10784 raft_consensus.cc:388] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:21.230685 10784 raft_consensus.cc:482] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:21.230801 10784 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:21.232143 10784 raft_consensus.cc:504] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27484d8abb864e2da97fd6353f869cb0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34527 } }
I20260807 00:02:21.232571 10784 leader_election.cc:302] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 27484d8abb864e2da97fd6353f869cb0; no voters:
I20260807 00:02:21.232957 10784 leader_election.cc:288] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:21.233109 10791 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:21.233366 10791 raft_consensus.cc:686] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [term 1 LEADER]: Becoming Leader. State: Replica: 27484d8abb864e2da97fd6353f869cb0, State: Running, Role: LEADER
I20260807 00:02:21.233788 10791 consensus_queue.cc:237] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27484d8abb864e2da97fd6353f869cb0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34527 } }
I20260807 00:02:21.234395 10784 sys_catalog.cc:564] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:21.235651 10794 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "27484d8abb864e2da97fd6353f869cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27484d8abb864e2da97fd6353f869cb0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34527 } } }
I20260807 00:02:21.235790 10794 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:21.236248 10807 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:21.236581 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.221856 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38464 (local address 127.30.204.126:34527)
0807 00:02:21.222358 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:21.222375 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:21.222416 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:02:21.223170 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:21.223192 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:21.223658 (+ 466us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:21.224023 (+ 365us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:21.224936 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.226012 (+ 1076us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.226921 (+ 909us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.227496 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.227672 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.230445 (+ 2773us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:21.230453 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:21.230472 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:21.230516 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:21.232642 (+ 2126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.233317 (+ 675us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.233326 (+ 9us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.233333 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.233533 (+ 200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.233898 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.233902 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.233904 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.234500 (+ 596us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:21.234702 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:21.235045 (+ 343us) server_negotiation.cc:299] Negotiation successful
0807 00:02:21.235413 (+ 368us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":136,"threads_started":1}
I20260807 00:02:21.237098 10795 sys_catalog.cc:455] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 27484d8abb864e2da97fd6353f869cb0. Latest consensus state: current_term: 1 leader_uuid: "27484d8abb864e2da97fd6353f869cb0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "27484d8abb864e2da97fd6353f869cb0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34527 } } }
I20260807 00:02:21.237241 10795 sys_catalog.cc:458] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:21.240222 10807 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:21.246687 10807 catalog_manager.cc:1347] Generated new cluster ID: 823821d615654981aa73b8d94e44b2d3
I20260807 00:02:21.246788 10807 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:21.282115 10807 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:21.283203 10807 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:21.289348 10807 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Generated new TSK 0
I20260807 00:02:21.290210 10807 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab.
Entry for principal oryx/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab.
I20260807 00:02:21.335052 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34527
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab
--principal=oryx/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:21.460783 10827 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:21.461082 10827 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:21.461127 10827 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:21.465384 10827 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:21.465468 10827 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:21.465510 10827 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:02:21.465544 10827 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:21.471148 10827 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab
--principal=oryx/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.10827
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:21.472486 10827 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:21.473570 10827 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:21.484019 10837 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:21.484023 10840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:21.484170 10827 server_base.cc:1034] running on GCE node
W20260807 00:02:21.484021 10835 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:21.484858 10827 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:21.485575 10827 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:21.486783 10827 hybrid_clock.cc:648] HybridClock initialized: now 1786060941486757 us; error 38 us; skew 500 ppm
Aug 07 00:02:21 dist-test-slave-pww0 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:21.489827 10827 init.cc:377] Logged in from keytab as oryx/127.30.204.65@KRBTEST.COM (short username oryx)
I20260807 00:02:21.490908 10827 webserver.cc:460] Webserver started at http://127.30.204.65:34841/ using document root <none> and password file <none>
I20260807 00:02:21.491547 10827 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:21.491641 10827 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:21.491868 10827 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:21.494139 10827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "b625a2cfbb47489fa03310583e7a0639"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "3dfff88a67310b481f09d3a38a7123b1"
server_key_iv: "63ee044a896d3246dac74bb81b2a0a04"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.494750 10827 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "b625a2cfbb47489fa03310583e7a0639"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "3dfff88a67310b481f09d3a38a7123b1"
server_key_iv: "63ee044a896d3246dac74bb81b2a0a04"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.499048 10827 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:21.501991 10853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:21.503432 10827 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:21.503628 10827 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "b625a2cfbb47489fa03310583e7a0639"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "3dfff88a67310b481f09d3a38a7123b1"
server_key_iv: "63ee044a896d3246dac74bb81b2a0a04"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.503755 10827 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:21.520987 10827 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:21.524472 10827 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:21.524669 10827 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:21.526384 10827 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:21.526445 10827 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:21.526490 10827 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:21.526505 10827 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:02:21 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:21.537364 10827 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:44655
I20260807 00:02:21.537448 11031 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:44655 every 8 connection(s)
I20260807 00:02:21.538359 10827 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:21.542033 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.527210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:48049 (local address 127.30.204.126:34527)
0807 00:02:21.527340 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:21.527344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:21.528548 (+ 1204us) server_negotiation.cc:407] Connection header received
0807 00:02:21.529628 (+ 1080us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:21.529632 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:21.529699 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:21.529789 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:21.531451 (+ 1662us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.532115 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.533386 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.533740 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.533865 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.537726 (+ 3861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:21.537730 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:21.537733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:21.537766 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:21.539641 (+ 1875us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.540323 (+ 682us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.540327 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.540328 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.540430 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.540864 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.540866 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.540868 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.541079 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:21.541159 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:21.541771 (+ 612us) server_negotiation.cc:299] Negotiation successful
0807 00:02:21.541900 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 00:02:21.542902 10866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.527845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34527 (local address 127.30.204.65:48049)
0807 00:02:21.528377 (+ 532us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:21.528425 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:21.529335 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:21.530248 (+ 913us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:21.530257 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:21.530685 (+ 428us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:21.531274 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.531290 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.532250 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.532254 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.533249 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.533262 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.533949 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.533955 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.534189 (+ 234us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.535539 (+ 1350us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:21.535566 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:21.537503 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:21.539864 (+ 2361us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.539871 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.539887 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.540189 (+ 302us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.540554 (+ 365us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.540558 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.540559 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.540758 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.541190 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:21.541198 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:21.541490 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 00:02:21.541667 (+ 177us) client_negotiation.cc:240] Negotiation successful
0807 00:02:21.541987 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":148,"threads_started":1}
I20260807 00:02:21.544085 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 10827
I20260807 00:02:21.544250 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:21.544423 11033 heartbeater.cc:347] Connected to a master server at 127.30.204.126:34527
I20260807 00:02:21.544574 11033 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:21.544612 31537 external_mini_cluster.cc:1428] Setting key 17d5d2a04d1b21623523f989a05b099b
I20260807 00:02:21.545217 11033 heartbeater.cc:511] Master 127.30.204.126:34527 requested a full tablet report, sending...
I20260807 00:02:21.546896 10721 ts_manager.cc:194] Registered new tserver with Master: b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655)
I20260807 00:02:21.548437 10721 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:48049
I20260807 00:02:21.557484 11042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.546699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34527 (local address 127.30.204.65:47827)
0807 00:02:21.547074 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:21.547088 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:21.547196 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:21.549944 (+ 2748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:21.549946 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:21.550186 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:21.550343 (+ 157us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.550352 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.551359 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.551363 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.552186 (+ 823us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.552199 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.552677 (+ 478us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.552682 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.552788 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.553352 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:21.553366 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:21.553732 (+ 366us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:21.555825 (+ 2093us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.555828 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.555829 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.556052 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.556404 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.556406 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.556408 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.556508 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.556986 (+ 478us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:21.556989 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:21.557088 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 00:02:21.557172 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:02:21.557311 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":112,"threads_started":1}
WARNING: no policy specified for oryx/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:21.557560 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.546781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:47827 (local address 127.30.204.126:34527)
0807 00:02:21.546902 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:21.546906 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:21.547172 (+ 266us) server_negotiation.cc:407] Connection header received
0807 00:02:21.547319 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:21.547322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:21.547367 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:21.547413 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:21.550493 (+ 3080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.551221 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.552327 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.552548 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.552630 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.553899 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:21.553903 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:21.553906 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:21.553940 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:21.555695 (+ 1755us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.556171 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.556175 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.556176 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.556287 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.556616 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.556619 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.556621 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.556877 (+ 256us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:21.556987 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:21.557266 (+ 279us) server_negotiation.cc:299] Negotiation successful
0807 00:02:21.557396 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab.
Entry for principal oryx/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab.
I20260807 00:02:21.578732 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34527
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab
--principal=oryx/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:21.695788 11049 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:21.696075 11049 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:21.696122 11049 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:21.700316 11049 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:21.700402 11049 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:21.700443 11049 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:02:21.700476 11049 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:21.706137 11049 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab
--principal=oryx/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11049
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:21.707436 11049 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:21.708671 11049 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:21.719369 11059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:21.719395 11058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:21.719571 11049 server_base.cc:1034] running on GCE node
W20260807 00:02:21.719825 11062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:21.720258 11049 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:21.720974 11049 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:21.722152 11049 hybrid_clock.cc:648] HybridClock initialized: now 1786060941722131 us; error 40 us; skew 500 ppm
Aug 07 00:02:21 dist-test-slave-pww0 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:21.725248 11049 init.cc:377] Logged in from keytab as oryx/127.30.204.66@KRBTEST.COM (short username oryx)
I20260807 00:02:21.726311 11049 webserver.cc:460] Webserver started at http://127.30.204.66:36009/ using document root <none> and password file <none>
I20260807 00:02:21.726945 11049 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:21.727021 11049 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:21.727260 11049 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:21.729599 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "9974f333337e4a399206e6ea902748ee"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "4d1d0e3f124b9078e3252edcb01cf99e"
server_key_iv: "8b3bef097cb5204a222641bb738025d8"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.730216 11049 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "9974f333337e4a399206e6ea902748ee"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "4d1d0e3f124b9078e3252edcb01cf99e"
server_key_iv: "8b3bef097cb5204a222641bb738025d8"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.734419 11049 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:21.737458 11070 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:21.738776 11049 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:21.738912 11049 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "9974f333337e4a399206e6ea902748ee"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "4d1d0e3f124b9078e3252edcb01cf99e"
server_key_iv: "8b3bef097cb5204a222641bb738025d8"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.739027 11049 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:21.758250 11049 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:21.761775 11049 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:21.761965 11049 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:21.763631 11049 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:21.763698 11049 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:21.763762 11049 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:21.763793 11049 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Aug 07 00:02:21 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:21.774742 11049 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:34053
I20260807 00:02:21.774777 11272 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:34053 every 8 connection(s)
I20260807 00:02:21.776005 11049 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:21.777100 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 11049
I20260807 00:02:21.777232 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:21.777586 31537 external_mini_cluster.cc:1428] Setting key 673724153861ba52c90f04f69a36d3b4
I20260807 00:02:21.779124 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.764599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:55091 (local address 127.30.204.126:34527)
0807 00:02:21.764740 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:21.764744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:21.766091 (+ 1347us) server_negotiation.cc:407] Connection header received
0807 00:02:21.767327 (+ 1236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:21.767332 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:21.767402 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:21.767557 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:21.769051 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.769613 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.770877 (+ 1264us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.771120 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.771221 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.774638 (+ 3417us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:21.774644 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:21.774646 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:21.774686 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:21.776421 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.777114 (+ 693us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.777118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.777120 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.777241 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.777708 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.777712 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.777714 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.777969 (+ 255us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:21.778051 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:21.778829 (+ 778us) server_negotiation.cc:299] Negotiation successful
0807 00:02:21.778993 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:02:21.780102 11087 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.765231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34527 (local address 127.30.204.66:55091)
0807 00:02:21.765872 (+ 641us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:21.765928 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:21.766997 (+ 1069us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:21.767753 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:21.767764 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:21.768276 (+ 512us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:21.768882 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.768898 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.769746 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.769750 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.770706 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.770720 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.771301 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.771305 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.771573 (+ 268us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.772769 (+ 1196us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:21.772787 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:21.774345 (+ 1558us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:21.776580 (+ 2235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.776586 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.776603 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.776989 (+ 386us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.777371 (+ 382us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.777375 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.777377 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.777581 (+ 204us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.778180 (+ 599us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:21.778187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:21.778480 (+ 293us) client_negotiation.cc:769] Sending connection context
0807 00:02:21.778730 (+ 250us) client_negotiation.cc:240] Negotiation successful
0807 00:02:21.779100 (+ 370us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":406,"thread_start_us":184,"threads_started":1}
I20260807 00:02:21.781664 11274 heartbeater.cc:347] Connected to a master server at 127.30.204.126:34527
I20260807 00:02:21.781848 11274 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:21.782503 11274 heartbeater.cc:511] Master 127.30.204.126:34527 requested a full tablet report, sending...
I20260807 00:02:21.783843 10720 ts_manager.cc:194] Registered new tserver with Master: 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053)
I20260807 00:02:21.784669 10720 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:55091
WARNING: no policy specified for oryx/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.67@KRBTEST.COM" created.
I20260807 00:02:21.793417 11286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.784029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34527 (local address 127.30.204.66:43547)
0807 00:02:21.785679 (+ 1650us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:21.785696 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:21.785808 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:21.786090 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:21.786093 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:21.786320 (+ 227us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:21.786442 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.786450 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.787569 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.787573 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.788205 (+ 632us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:21.788215 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.788880 (+ 665us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.788883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:21.789005 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.789646 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:21.789662 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:21.790025 (+ 363us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:21.791740 (+ 1715us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.791743 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.791744 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.791996 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.792305 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:21.792307 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:21.792308 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:21.792389 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:21.792929 (+ 540us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:21.792931 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:21.793027 (+ 96us) client_negotiation.cc:769] Sending connection context
0807 00:02:21.793111 (+ 84us) client_negotiation.cc:240] Negotiation successful
0807 00:02:21.793248 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1546,"thread_start_us":135,"threads_started":1}
I20260807 00:02:21.793483 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.784048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:43547 (local address 127.30.204.126:34527)
0807 00:02:21.784185 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:21.784190 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:21.785790 (+ 1600us) server_negotiation.cc:407] Connection header received
0807 00:02:21.785905 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:21.785908 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:21.785966 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:21.786059 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:21.786571 (+ 512us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.787386 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.788339 (+ 953us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:21.788726 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:21.788846 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:21.790191 (+ 1345us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:21.790194 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:21.790196 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:21.790224 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:21.791632 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.792118 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.792122 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.792124 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.792221 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:21.792486 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:21.792490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:21.792492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:21.792826 (+ 334us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:21.792930 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:21.793194 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 00:02:21.793323 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab.
Entry for principal oryx/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab.
I20260807 00:02:21.811148 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34527
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab
--principal=oryx/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41949
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:21.935393 11290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:21.935738 11290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:21.935786 11290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:21.940083 11290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:21.940163 11290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:21.940214 11290 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260807 00:02:21.940248 11290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:21.946444 11290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41949
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab
--principal=oryx/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.30.204.126:34527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:21.947671 11290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:21.948797 11290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:21.959265 11300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:21.959286 11305 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:21.959301 11301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:21.959856 11290 server_base.cc:1034] running on GCE node
I20260807 00:02:21.960341 11290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:21.961073 11290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:21.962244 11290 hybrid_clock.cc:648] HybridClock initialized: now 1786060941962223 us; error 40 us; skew 500 ppm
Aug 07 00:02:21 dist-test-slave-pww0 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:21.965351 11290 init.cc:377] Logged in from keytab as oryx/127.30.204.67@KRBTEST.COM (short username oryx)
I20260807 00:02:21.966440 11290 webserver.cc:460] Webserver started at http://127.30.204.67:43525/ using document root <none> and password file <none>
I20260807 00:02:21.967059 11290 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:21.967137 11290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:21.967386 11290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:21.969722 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "78fb8157baad4f838c27aae825c67270"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "d005b0bc944b3bcab9c16fe514659fed"
server_key_iv: "9af47c8e451a5109e69dce8d0444aa11"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.970373 11290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "78fb8157baad4f838c27aae825c67270"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "d005b0bc944b3bcab9c16fe514659fed"
server_key_iv: "9af47c8e451a5109e69dce8d0444aa11"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.974745 11290 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:21.978583 11314 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:21.980265 11290 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:02:21.980415 11290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "78fb8157baad4f838c27aae825c67270"
format_stamp: "Formatted at 2026-08-07 00:02:21 on dist-test-slave-pww0"
server_key: "d005b0bc944b3bcab9c16fe514659fed"
server_key_iv: "9af47c8e451a5109e69dce8d0444aa11"
server_key_version: "encryptionkey@0"
I20260807 00:02:21.980564 11290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:21.993670 11290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:21.997076 11290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:21.997288 11290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:21.998911 11290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:21.998968 11290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:21.999014 11290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:21.999044 11290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:22.010412 11290 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:40065
Aug 07 00:02:22 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:22.010445 11513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:40065 every 8 connection(s)
I20260807 00:02:22.011657 11290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:22.015269 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:21.999919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:35333 (local address 127.30.204.126:34527)
0807 00:02:22.000042 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.000047 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.001339 (+ 1292us) server_negotiation.cc:407] Connection header received
0807 00:02:22.002835 (+ 1496us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.002839 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.002906 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.003001 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:22.004834 (+ 1833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.005690 (+ 856us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.007566 (+ 1876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.007852 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.007985 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.011283 (+ 3298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:22.011287 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:22.011289 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:22.011322 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:22.013112 (+ 1790us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.013660 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.013664 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.013666 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.013756 (+ 90us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.014097 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.014100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.014102 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.014317 (+ 215us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:22.014405 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.014944 (+ 539us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.015085 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:02:22.016253 11327 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.000565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34527 (local address 127.30.204.67:35333)
0807 00:02:22.001165 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.001214 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.002561 (+ 1347us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.003231 (+ 670us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.003242 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.003841 (+ 599us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:22.004629 (+ 788us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.004655 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.005873 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.005877 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.007361 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.007375 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.008035 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.008040 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.008335 (+ 295us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.009097 (+ 762us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:22.009116 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:22.011010 (+ 1894us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:22.013235 (+ 2225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:22.013243 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:22.013262 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:22.013543 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:22.013846 (+ 303us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:22.013849 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:22.013851 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:22.014000 (+ 149us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:22.014419 (+ 419us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:22.014427 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:22.014695 (+ 268us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.014884 (+ 189us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.015246 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":138,"threads_started":1}
I20260807 00:02:22.017549 11517 heartbeater.cc:347] Connected to a master server at 127.30.204.126:34527
I20260807 00:02:22.017724 11517 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:22.018383 11517 heartbeater.cc:511] Master 127.30.204.126:34527 requested a full tablet report, sending...
I20260807 00:02:22.019662 10721 ts_manager.cc:194] Registered new tserver with Master: 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:22.019721 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 11290
I20260807 00:02:22.019834 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:22.020176 31537 external_mini_cluster.cc:1428] Setting key fa2f9a96be6111e093eb45cf3e4fb5c7
I20260807 00:02:22.020465 10721 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:35333
I20260807 00:02:22.022468 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:02:22.029160 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.020211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:40213 (local address 127.30.204.126:34527)
0807 00:02:22.020340 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.020344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.020677 (+ 333us) server_negotiation.cc:407] Connection header received
0807 00:02:22.020944 (+ 267us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.020947 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.020995 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.021081 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:22.021562 (+ 481us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.022286 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.024262 (+ 1976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.024568 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.024682 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.025894 (+ 1212us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:22.025897 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:22.025899 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:22.025925 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:22.027441 (+ 1516us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.027935 (+ 494us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.027938 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.027940 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.028026 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.028295 (+ 269us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.028298 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.028300 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.028527 (+ 227us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:22.028624 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.028877 (+ 253us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.028992 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 00:02:22.029171 11525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.020133 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34527 (local address 127.30.204.67:40213)
0807 00:02:22.020591 (+ 458us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.020606 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.020837 (+ 231us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.021097 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.021099 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.021323 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:22.021463 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.021473 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.023523 (+ 2050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.023527 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.024126 (+ 599us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.024137 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.024675 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.024678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.024765 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.025294 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:22.025310 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:22.025745 (+ 435us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:22.027578 (+ 1833us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:22.027581 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:22.027583 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:22.027830 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:22.028113 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:22.028116 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:22.028118 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:22.028200 (+ 82us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:22.028614 (+ 414us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:22.028617 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:22.028718 (+ 101us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.028806 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.028971 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"thread_start_us":154,"threads_started":1}
Aug 07 00:02:22 dist-test-slave-pww0 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Aug 07 00:02:22 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:22 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:02:22.051115 10788 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:22.044339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43396 (local address 127.30.204.126:34527)
0807 00:02:22.044480 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.044484 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.044499 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:22.044563 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.044567 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.044621 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.044698 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:22.045555 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.046157 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.047102 (+ 945us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.047345 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.047431 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.050941 (+ 3510us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43396: BlockingRecv error: recv got EOF from 127.0.0.1:43396 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Aug 07 00:02:22 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060942, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:22.061870 10788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.052863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43410 (local address 127.30.204.126:34527)
0807 00:02:22.053026 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.053031 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.053174 (+ 143us) server_negotiation.cc:407] Connection header received
0807 00:02:22.053314 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.053317 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.053369 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.053437 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:22.054187 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.054712 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.055569 (+ 857us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.055816 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.055912 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.058384 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:22.058387 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:22.058389 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:22.058423 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:22.059839 (+ 1416us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.060315 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.060318 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.060319 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.060413 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.060819 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.060822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.060824 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.061083 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:22.061232 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.061544 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.061677 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 00:02:22.065356 10721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:22.068643 10721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:22.084704 11556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.078322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40065 (local address 127.0.0.1:42070)
0807 00:02:22.079605 (+ 1283us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.079617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.079707 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.080526 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.080529 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.080547 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:22.080617 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.080635 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.081978 (+ 1343us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.081981 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.083141 (+ 1160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.083151 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.084100 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.084106 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.084231 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.084277 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.084390 (+ 113us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.084531 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1227,"mutex_wait_us":35,"thread_start_us":83,"threads_started":1}
I20260807 00:02:22.084714 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.078996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42070 (local address 127.30.204.67:40065)
0807 00:02:22.079870 (+ 874us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.079886 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.079909 (+ 23us) server_negotiation.cc:407] Connection header received
0807 00:02:22.079982 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.079986 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.080158 (+ 172us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.080370 (+ 212us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:22.080761 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.081868 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.083265 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.083928 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.084034 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.084227 (+ 193us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.084437 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.084538 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":358,"thread_start_us":111,"threads_started":1}
I20260807 00:02:22.085016 11553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.078097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44655 (local address 127.0.0.1:58426)
0807 00:02:22.078749 (+ 652us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.078807 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.078953 (+ 146us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.080122 (+ 1169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.080129 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.080153 (+ 24us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:22.080331 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.080338 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.081510 (+ 1172us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.081513 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.083175 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.083183 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.084504 (+ 1321us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.084508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.084691 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.084721 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.084788 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.084870 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":463,"mutex_wait_us":325,"spinlock_wait_cycles":6528,"thread_start_us":116,"threads_started":1}
I20260807 00:02:22.085435 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.078996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58426 (local address 127.30.204.65:44655)
0807 00:02:22.079548 (+ 552us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.079554 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.079577 (+ 23us) server_negotiation.cc:407] Connection header received
0807 00:02:22.079657 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.079662 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.079951 (+ 289us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.080123 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:22.080457 (+ 334us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.081401 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.083304 (+ 1903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.084172 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.084281 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.084435 (+ 154us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.085194 (+ 759us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.085305 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":407,"thread_start_us":111,"threads_started":1}
I20260807 00:02:22.085448 11555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.078041 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:34053 (local address 127.0.0.1:43944)
0807 00:02:22.079210 (+ 1169us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.079223 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.079339 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.080155 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.080158 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.080178 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:22.080333 (+ 155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.080342 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.081841 (+ 1499us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.081844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.083145 (+ 1301us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.083157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.084504 (+ 1347us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.084508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.084696 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.084719 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.085249 (+ 530us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.085332 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":963,"mutex_wait_us":71,"thread_start_us":145,"threads_started":1}
I20260807 00:02:22.085924 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.078233 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43944 (local address 127.30.204.66:34053)
0807 00:02:22.079229 (+ 996us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.079243 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.079436 (+ 193us) server_negotiation.cc:407] Connection header received
0807 00:02:22.079564 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.079570 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.079820 (+ 250us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.079979 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:22.080466 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.081714 (+ 1248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.083303 (+ 1589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.084172 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.084281 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.084450 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.085704 (+ 1254us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.085812 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":880,"thread_start_us":103,"threads_started":1}
I20260807 00:02:22.088487 11150 tablet_service.cc:1467] Processing CreateTablet for tablet 152e0faef03e49d8a6542d3b4d4b9925 (DEFAULT_TABLE table=test-table [id=287cbf3c1b9940a68945a36a82e29c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:22.088518 11405 tablet_service.cc:1467] Processing CreateTablet for tablet 152e0faef03e49d8a6542d3b4d4b9925 (DEFAULT_TABLE table=test-table [id=287cbf3c1b9940a68945a36a82e29c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:22.088515 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 152e0faef03e49d8a6542d3b4d4b9925 (DEFAULT_TABLE table=test-table [id=287cbf3c1b9940a68945a36a82e29c3e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:22.089488 11150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 152e0faef03e49d8a6542d3b4d4b9925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.089581 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 152e0faef03e49d8a6542d3b4d4b9925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.089581 11405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 152e0faef03e49d8a6542d3b4d4b9925. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.097283 11562 tablet_bootstrap.cc:492] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Bootstrap starting.
I20260807 00:02:22.098194 11563 tablet_bootstrap.cc:492] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Bootstrap starting.
I20260807 00:02:22.098194 11564 tablet_bootstrap.cc:492] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Bootstrap starting.
I20260807 00:02:22.100168 11562 tablet_bootstrap.cc:654] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.100404 11564 tablet_bootstrap.cc:654] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.100404 11563 tablet_bootstrap.cc:654] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.100986 11562 log.cc:824] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:22.101269 11564 log.cc:824] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:22.101267 11563 log.cc:824] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:22.103365 11564 tablet_bootstrap.cc:492] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: No bootstrap required, opened a new log
I20260807 00:02:22.103365 11563 tablet_bootstrap.cc:492] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: No bootstrap required, opened a new log
I20260807 00:02:22.103365 11562 tablet_bootstrap.cc:492] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: No bootstrap required, opened a new log
I20260807 00:02:22.103626 11563 ts_tablet_manager.cc:1397] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:02:22.103623 11562 ts_tablet_manager.cc:1397] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:02:22.103670 11564 ts_tablet_manager.cc:1397] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:02:22.107039 11564 raft_consensus.cc:348] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.107553 11564 raft_consensus.cc:374] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.107630 11564 raft_consensus.cc:729] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb8157baad4f838c27aae825c67270, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.107676 11562 raft_consensus.cc:348] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.107973 11562 raft_consensus.cc:374] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.108035 11562 raft_consensus.cc:729] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.108173 11564 consensus_queue.cc:260] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.108232 11563 raft_consensus.cc:348] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.108515 11563 raft_consensus.cc:374] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.108497 11562 consensus_queue.cc:260] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.108577 11563 raft_consensus.cc:729] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b625a2cfbb47489fa03310583e7a0639, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.109010 11563 consensus_queue.cc:260] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.109416 11564 ts_tablet_manager.cc:1428] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:02:22.109560 11562 ts_tablet_manager.cc:1428] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:02:22.109807 11517 heartbeater.cc:503] Master 127.30.204.126:34527 was elected leader, sending a full tablet report...
I20260807 00:02:22.109899 11274 heartbeater.cc:503] Master 127.30.204.126:34527 was elected leader, sending a full tablet report...
I20260807 00:02:22.109997 11563 ts_tablet_manager.cc:1428] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:02:22.110317 11033 heartbeater.cc:503] Master 127.30.204.126:34527 was elected leader, sending a full tablet report...
W20260807 00:02:22.263023 11518 tablet.cc:2369] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:02:22.278324 11275 tablet.cc:2369] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:02:22.289960 11034 tablet.cc:2369] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:02:22.369642 11573 raft_consensus.cc:482] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:22.369895 11573 raft_consensus.cc:504] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.371191 11573 leader_election.cc:288] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:22.375511 11558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.371639 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:41267 (local address 127.30.204.67:40065)
0807 00:02:22.371850 (+ 211us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.371855 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.372001 (+ 146us) server_negotiation.cc:407] Connection header received
0807 00:02:22.372135 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.372138 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.372184 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.372256 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:22.372565 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.373268 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.374476 (+ 1208us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.374892 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.375018 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.375075 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.375272 (+ 197us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.375354 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":117}
I20260807 00:02:22.375531 11576 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.371546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40065 (local address 127.30.204.66:41267)
0807 00:02:22.371902 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.371916 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.372030 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.372278 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.372281 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.372301 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:22.372440 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.372446 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.373370 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.373373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.374346 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.374356 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.375007 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.375010 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.375145 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.375168 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.375236 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.375324 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":302,"mutex_wait_us":122,"thread_start_us":70,"threads_started":1}
I20260807 00:02:22.375526 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.371633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:36665 (local address 127.30.204.65:44655)
0807 00:02:22.371781 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.371786 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.371940 (+ 154us) server_negotiation.cc:407] Connection header received
0807 00:02:22.372127 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.372131 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.372185 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.372234 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:22.372679 (+ 445us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.373263 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.374469 (+ 1206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.374897 (+ 428us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.375007 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.375070 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.375298 (+ 228us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.375380 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 00:02:22.375547 11577 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.371449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44655 (local address 127.30.204.66:36665)
0807 00:02:22.371835 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.371851 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.371971 (+ 120us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.372392 (+ 421us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.372394 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.372412 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:22.372524 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.372530 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.373382 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.373385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.374341 (+ 956us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.374351 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.375011 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.375014 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.375171 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.375188 (+ 17us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.375252 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.375333 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"mutex_wait_us":78,"thread_start_us":104,"threads_started":1}
I20260807 00:02:22.376526 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "152e0faef03e49d8a6542d3b4d4b9925" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639" is_pre_election: true
I20260807 00:02:22.376528 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "152e0faef03e49d8a6542d3b4d4b9925" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270" is_pre_election: true
I20260807 00:02:22.376914 11439 raft_consensus.cc:2454] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 0.
I20260807 00:02:22.376914 10959 raft_consensus.cc:2454] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 0.
I20260807 00:02:22.377488 11078 leader_election.cc:302] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9974f333337e4a399206e6ea902748ee, b625a2cfbb47489fa03310583e7a0639; no voters:
I20260807 00:02:22.377839 11573 raft_consensus.cc:2781] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:22.377941 11573 raft_consensus.cc:482] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:22.377985 11573 raft_consensus.cc:3037] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.379256 11573 raft_consensus.cc:504] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.379701 11573 leader_election.cc:288] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 election: Requested vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:22.380193 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "152e0faef03e49d8a6542d3b4d4b9925" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639"
I20260807 00:02:22.380213 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "152e0faef03e49d8a6542d3b4d4b9925" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270"
I20260807 00:02:22.380344 11439 raft_consensus.cc:3037] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.380329 10959 raft_consensus.cc:3037] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.381500 10959 raft_consensus.cc:2454] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 1.
I20260807 00:02:22.381934 11439 raft_consensus.cc:2454] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 1.
I20260807 00:02:22.381965 11078 leader_election.cc:302] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9974f333337e4a399206e6ea902748ee, b625a2cfbb47489fa03310583e7a0639; no voters:
I20260807 00:02:22.382169 11573 raft_consensus.cc:2781] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:22.382370 11573 raft_consensus.cc:686] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 1 LEADER]: Becoming Leader. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Running, Role: LEADER
I20260807 00:02:22.382761 11573 consensus_queue.cc:237] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.386376 10719 catalog_manager.cc:5495] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 9974f333337e4a399206e6ea902748ee (127.30.204.66). New cstate: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } health_report { overall_health: UNKNOWN } } }
I20260807 00:02:22.453612 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.449710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43956 (local address 127.30.204.66:34053)
0807 00:02:22.449858 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.449862 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.449878 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:22.449993 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.449995 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.450069 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.450180 (+ 111us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:22.450495 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.451055 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.452042 (+ 987us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.452299 (+ 257us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.452397 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.452711 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.453188 (+ 477us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.453303 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.453381 (+ 78us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.453455 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 00:02:22.460333 10959 raft_consensus.cc:1261] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Refusing update from remote peer 9974f333337e4a399206e6ea902748ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:22.460332 11439 raft_consensus.cc:1261] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Refusing update from remote peer 9974f333337e4a399206e6ea902748ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:22.461323 11573 consensus_queue.cc:1035] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:22.461587 11578 consensus_queue.cc:1035] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:22.469739 11584 mvcc.cc:205] Tried to move back new op lower bound from 7315705620308926464 to 7315705620003188736. Current Snapshot: MvccSnapshot[applied={T|T < 7315705620308926464}]
I20260807 00:02:22.471446 11589 mvcc.cc:205] Tried to move back new op lower bound from 7315705620308926464 to 7315705620003188736. Current Snapshot: MvccSnapshot[applied={T|T < 7315705620308926464}]
I20260807 00:02:22.479666 10719 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:22.479869 10719 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:22.482709 10719 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Sending DeleteTablet for 3 replicas of tablet 152e0faef03e49d8a6542d3b4d4b9925
I20260807 00:02:22.483677 11150 tablet_service.cc:1514] Processing DeleteTablet for tablet 152e0faef03e49d8a6542d3b4d4b9925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:22 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:43944
I20260807 00:02:22.483701 11405 tablet_service.cc:1514] Processing DeleteTablet for tablet 152e0faef03e49d8a6542d3b4d4b9925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:22 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:42070
I20260807 00:02:22.483685 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet 152e0faef03e49d8a6542d3b4d4b9925 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:22 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58426
I20260807 00:02:22.484465 11608 tablet_replica.cc:330] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: stopping tablet replica
I20260807 00:02:22.484576 11607 tablet_replica.cc:330] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: stopping tablet replica
I20260807 00:02:22.484864 11608 raft_consensus.cc:2229] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:22.485229 11607 raft_consensus.cc:2229] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:22.485270 11608 raft_consensus.cc:2258] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:22.485409 10719 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43410:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260807 00:02:22.485548 11607 raft_consensus.cc:2258] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 00:02:22.485903 10719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:22.486984 11608 ts_tablet_manager.cc:1905] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:22.487679 11607 ts_tablet_manager.cc:1905] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:22.489997 11608 ts_tablet_manager.cc:1918] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:22.490118 11608 log.cc:1196] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/152e0faef03e49d8a6542d3b4d4b9925
I20260807 00:02:22.490384 11607 ts_tablet_manager.cc:1918] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:22.490478 11608 ts_tablet_manager.cc:1939] T 152e0faef03e49d8a6542d3b4d4b9925 P b625a2cfbb47489fa03310583e7a0639: Deleting consensus metadata
I20260807 00:02:22.490486 11607 log.cc:1196] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/152e0faef03e49d8a6542d3b4d4b9925
I20260807 00:02:22.490787 11607 ts_tablet_manager.cc:1939] T 152e0faef03e49d8a6542d3b4d4b9925 P 78fb8157baad4f838c27aae825c67270: Deleting consensus metadata
I20260807 00:02:22.491940 10701 catalog_manager.cc:4841] TS 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065): tablet 152e0faef03e49d8a6542d3b4d4b9925 (table test-table [id=287cbf3c1b9940a68945a36a82e29c3e]) successfully deleted
I20260807 00:02:22.492429 11605 tablet_replica.cc:330] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: stopping tablet replica
I20260807 00:02:22.492626 11405 tablet_service.cc:1467] Processing CreateTablet for tablet 323c3bc0d2634775b26644f393a3f2d2 (DEFAULT_TABLE table=test-table [id=cf422f7bb666435aa08b417ab62892bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:22.492630 11150 tablet_service.cc:1467] Processing CreateTablet for tablet 323c3bc0d2634775b26644f393a3f2d2 (DEFAULT_TABLE table=test-table [id=cf422f7bb666435aa08b417ab62892bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:22.492817 11605 raft_consensus.cc:2229] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:22.492903 11405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323c3bc0d2634775b26644f393a3f2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.492926 11150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323c3bc0d2634775b26644f393a3f2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.493211 11605 raft_consensus.cc:2258] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:22.494877 11564 tablet_bootstrap.cc:492] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: Bootstrap starting.
I20260807 00:02:22.495663 11562 tablet_bootstrap.cc:492] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: Bootstrap starting.
I20260807 00:02:22.495767 10704 catalog_manager.cc:4841] TS b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655): tablet 152e0faef03e49d8a6542d3b4d4b9925 (table test-table [id=287cbf3c1b9940a68945a36a82e29c3e]) successfully deleted
I20260807 00:02:22.496181 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 323c3bc0d2634775b26644f393a3f2d2 (DEFAULT_TABLE table=test-table [id=cf422f7bb666435aa08b417ab62892bc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:22.496466 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 323c3bc0d2634775b26644f393a3f2d2. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.496634 11562 tablet_bootstrap.cc:654] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.497640 11564 tablet_bootstrap.cc:654] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.498550 11562 tablet_bootstrap.cc:492] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: No bootstrap required, opened a new log
I20260807 00:02:22.498637 11562 ts_tablet_manager.cc:1397] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:22.498816 11563 tablet_bootstrap.cc:492] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: Bootstrap starting.
I20260807 00:02:22.499097 11562 raft_consensus.cc:348] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.499200 11562 raft_consensus.cc:374] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.499231 11562 raft_consensus.cc:729] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.499359 11562 consensus_queue.cc:260] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.499862 11563 tablet_bootstrap.cc:654] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.499989 11564 tablet_bootstrap.cc:492] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: No bootstrap required, opened a new log
I20260807 00:02:22.500080 11564 ts_tablet_manager.cc:1397] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: Time spent bootstrapping tablet: real 0.005s user 0.002s sys 0.000s
I20260807 00:02:22.500540 11564 raft_consensus.cc:348] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.500643 11564 raft_consensus.cc:374] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.500672 11564 raft_consensus.cc:729] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb8157baad4f838c27aae825c67270, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.500805 11564 consensus_queue.cc:260] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.501087 11562 ts_tablet_manager.cc:1428] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:02:22.501124 11563 tablet_bootstrap.cc:492] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: No bootstrap required, opened a new log
I20260807 00:02:22.501204 11563 ts_tablet_manager.cc:1397] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260807 00:02:22.501394 11564 ts_tablet_manager.cc:1428] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:22.501935 11563 raft_consensus.cc:348] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.502987 11605 ts_tablet_manager.cc:1905] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:22.502060 11563 raft_consensus.cc:374] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.503445 11563 raft_consensus.cc:729] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b625a2cfbb47489fa03310583e7a0639, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.503643 11563 consensus_queue.cc:260] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.504160 11563 ts_tablet_manager.cc:1428] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260807 00:02:22.505771 11605 ts_tablet_manager.cc:1918] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:22.505890 11605 log.cc:1196] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/152e0faef03e49d8a6542d3b4d4b9925
I20260807 00:02:22.506212 11605 ts_tablet_manager.cc:1939] T 152e0faef03e49d8a6542d3b4d4b9925 P 9974f333337e4a399206e6ea902748ee: Deleting consensus metadata
I20260807 00:02:22.507433 10703 catalog_manager.cc:4841] TS 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053): tablet 152e0faef03e49d8a6542d3b4d4b9925 (table test-table [id=287cbf3c1b9940a68945a36a82e29c3e]) successfully deleted
I20260807 00:02:22.726711 11572 raft_consensus.cc:482] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:22.726892 11572 raft_consensus.cc:504] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.728155 11572 leader_election.cc:288] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053)
I20260807 00:02:22.732043 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.728534 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:51169 (local address 127.30.204.65:44655)
0807 00:02:22.728707 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.728711 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.728866 (+ 155us) server_negotiation.cc:407] Connection header received
0807 00:02:22.729007 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.729010 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.729053 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.729114 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:22.729435 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.729942 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.731067 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.731483 (+ 416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.731580 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.731646 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.731813 (+ 167us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.731902 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:02:22.732055 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.728590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:36337 (local address 127.30.204.66:34053)
0807 00:02:22.728718 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.728723 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.728889 (+ 166us) server_negotiation.cc:407] Connection header received
0807 00:02:22.729011 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.729013 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.729060 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.729132 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:22.729456 (+ 324us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.729941 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.731081 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.731526 (+ 445us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.731654 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.731710 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.731872 (+ 162us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.731952 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:02:22.732052 11620 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.728478 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:34053 (local address 127.30.204.67:36337)
0807 00:02:22.728794 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.728805 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.728913 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.729173 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.729175 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.729220 (+ 45us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:22.729343 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.729348 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.730049 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.730052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.730950 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.730960 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.731657 (+ 697us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.731661 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.731749 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.731769 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.731844 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.731910 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":263,"mutex_wait_us":47,"spinlock_wait_cycles":2688,"thread_start_us":69,"threads_started":1}
I20260807 00:02:22.732074 11621 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.728427 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44655 (local address 127.30.204.67:51169)
0807 00:02:22.728774 (+ 347us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.728786 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.728898 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.729146 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.729149 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.729164 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:22.729329 (+ 165us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.729334 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.730054 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.730058 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.730946 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.730956 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.731588 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.731593 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.731690 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.731714 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.731787 (+ 73us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.731863 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":282,"mutex_wait_us":62,"spinlock_wait_cycles":17792,"thread_start_us":95,"threads_started":1}
I20260807 00:02:22.732573 11191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323c3bc0d2634775b26644f393a3f2d2" candidate_uuid: "78fb8157baad4f838c27aae825c67270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9974f333337e4a399206e6ea902748ee" is_pre_election: true
I20260807 00:02:22.732730 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323c3bc0d2634775b26644f393a3f2d2" candidate_uuid: "78fb8157baad4f838c27aae825c67270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639" is_pre_election: true
I20260807 00:02:22.732861 10959 raft_consensus.cc:2454] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78fb8157baad4f838c27aae825c67270 in term 0.
I20260807 00:02:22.732869 11191 raft_consensus.cc:2454] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 78fb8157baad4f838c27aae825c67270 in term 0.
I20260807 00:02:22.733301 11318 leader_election.cc:302] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 78fb8157baad4f838c27aae825c67270, 9974f333337e4a399206e6ea902748ee, b625a2cfbb47489fa03310583e7a0639; no voters:
I20260807 00:02:22.733569 11572 raft_consensus.cc:2781] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:22.733664 11572 raft_consensus.cc:482] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:22.733716 11572 raft_consensus.cc:3037] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.734851 11572 raft_consensus.cc:504] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.735266 11572 leader_election.cc:288] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [CANDIDATE]: Term 1 election: Requested vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053)
I20260807 00:02:22.735746 11191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323c3bc0d2634775b26644f393a3f2d2" candidate_uuid: "78fb8157baad4f838c27aae825c67270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9974f333337e4a399206e6ea902748ee"
I20260807 00:02:22.735747 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "323c3bc0d2634775b26644f393a3f2d2" candidate_uuid: "78fb8157baad4f838c27aae825c67270" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639"
I20260807 00:02:22.735889 11191 raft_consensus.cc:3037] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.735889 10959 raft_consensus.cc:3037] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.737066 10959 raft_consensus.cc:2454] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fb8157baad4f838c27aae825c67270 in term 1.
I20260807 00:02:22.737066 11191 raft_consensus.cc:2454] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 78fb8157baad4f838c27aae825c67270 in term 1.
I20260807 00:02:22.737398 11318 leader_election.cc:302] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb8157baad4f838c27aae825c67270, 9974f333337e4a399206e6ea902748ee; no voters:
I20260807 00:02:22.737586 11572 raft_consensus.cc:2781] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:22.737768 11572 raft_consensus.cc:686] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 1 LEADER]: Becoming Leader. State: Replica: 78fb8157baad4f838c27aae825c67270, State: Running, Role: LEADER
I20260807 00:02:22.738008 11572 consensus_queue.cc:237] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.740506 10719 catalog_manager.cc:5495] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 reported cstate change: term changed from 0 to 1, leader changed from <none> to 78fb8157baad4f838c27aae825c67270 (127.30.204.67). New cstate: current_term: 1 leader_uuid: "78fb8157baad4f838c27aae825c67270" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } health_report { overall_health: HEALTHY } } }
Aug 07 00:02:22 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060941, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.126@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:22.838963 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.829496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34527 (local address 127.0.0.1:43418)
0807 00:02:22.829882 (+ 386us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.829897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.830018 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.830287 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.830289 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.830704 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:22.830815 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.830822 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.831670 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.831673 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.832268 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.832278 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.832780 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.832783 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.832855 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.833522 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:22.833535 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:22.835176 (+ 1641us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:22.837187 (+ 2011us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:22.837202 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:22.837208 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:22.837484 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:22.837853 (+ 369us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:22.837855 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:22.837857 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:22.837957 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:22.838418 (+ 461us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:22.838426 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:22.838555 (+ 129us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.838662 (+ 107us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.838808 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":285,"thread_start_us":148,"threads_started":1}
I20260807 00:02:22.839059 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.829590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43418 (local address 127.30.204.126:34527)
0807 00:02:22.829905 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.829913 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.829970 (+ 57us) server_negotiation.cc:407] Connection header received
0807 00:02:22.830110 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.830113 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.830162 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.830245 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:22.830945 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.831540 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.832400 (+ 860us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.832643 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.832733 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.835359 (+ 2626us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:22.835362 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:22.835364 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:22.835388 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:22.837068 (+ 1680us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.837662 (+ 594us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.837665 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.837667 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.837758 (+ 91us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:22.838057 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:22.838059 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:22.838061 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:22.838279 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:22.838362 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.838770 (+ 408us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.838900 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":143,"threads_started":1}
I20260807 00:02:22.843746 10719 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:43418:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260807 00:02:22.844648 10719 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:22.850059 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 6f338e79c67d409988cffea9a3c781de (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=635d26a5a0c84b568d78896b1806f49a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:02:22.850311 11405 tablet_service.cc:1467] Processing CreateTablet for tablet 6f338e79c67d409988cffea9a3c781de (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=635d26a5a0c84b568d78896b1806f49a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:02:22.850580 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f338e79c67d409988cffea9a3c781de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.850673 11405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f338e79c67d409988cffea9a3c781de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.852094 11150 tablet_service.cc:1467] Processing CreateTablet for tablet 6f338e79c67d409988cffea9a3c781de (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=635d26a5a0c84b568d78896b1806f49a]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260807 00:02:22.852487 11150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6f338e79c67d409988cffea9a3c781de. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:22.853861 11564 tablet_bootstrap.cc:492] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: Bootstrap starting.
I20260807 00:02:22.854939 11564 tablet_bootstrap.cc:654] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.855489 11562 tablet_bootstrap.cc:492] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: Bootstrap starting.
I20260807 00:02:22.855763 11563 tablet_bootstrap.cc:492] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: Bootstrap starting.
I20260807 00:02:22.856348 11562 tablet_bootstrap.cc:654] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.856731 11563 tablet_bootstrap.cc:654] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:22.857915 11564 tablet_bootstrap.cc:492] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: No bootstrap required, opened a new log
I20260807 00:02:22.858013 11564 ts_tablet_manager.cc:1397] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260807 00:02:22.858100 11563 tablet_bootstrap.cc:492] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: No bootstrap required, opened a new log
I20260807 00:02:22.858101 11562 tablet_bootstrap.cc:492] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: No bootstrap required, opened a new log
I20260807 00:02:22.858201 11562 ts_tablet_manager.cc:1397] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:22.858212 11563 ts_tablet_manager.cc:1397] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260807 00:02:22.858489 11564 raft_consensus.cc:348] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.858613 11564 raft_consensus.cc:374] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.858644 11564 raft_consensus.cc:729] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb8157baad4f838c27aae825c67270, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.858633 11563 raft_consensus.cc:348] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.858633 11562 raft_consensus.cc:348] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.858759 11562 raft_consensus.cc:374] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.858759 11563 raft_consensus.cc:374] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:22.858794 11562 raft_consensus.cc:729] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.858794 11563 raft_consensus.cc:729] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b625a2cfbb47489fa03310583e7a0639, State: Initialized, Role: FOLLOWER
I20260807 00:02:22.858786 11564 consensus_queue.cc:260] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.858924 11562 consensus_queue.cc:260] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.858924 11563 consensus_queue.cc:260] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.859072 11564 ts_tablet_manager.cc:1428] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:22.859040 11572 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.859194 11572 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:02:22.859190 11562 ts_tablet_manager.cc:1428] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:22.859165 11575 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.859324 11575 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:02:22.859993 11563 ts_tablet_manager.cc:1428] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: Time spent starting tablet: real 0.002s user 0.001s sys 0.000s
I20260807 00:02:22.860133 11573 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.860268 11573 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:02:22.871860 11573 raft_consensus.cc:482] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:22.872005 11573 raft_consensus.cc:504] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.872399 11573 leader_election.cc:288] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:22.872900 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f338e79c67d409988cffea9a3c781de" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270" is_pre_election: true
I20260807 00:02:22.872908 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f338e79c67d409988cffea9a3c781de" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639" is_pre_election: true
I20260807 00:02:22.873059 11439 raft_consensus.cc:2454] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 0.
I20260807 00:02:22.873059 10959 raft_consensus.cc:2454] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 0.
I20260807 00:02:22.873411 11075 leader_election.cc:302] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb8157baad4f838c27aae825c67270, 9974f333337e4a399206e6ea902748ee; no voters:
I20260807 00:02:22.873643 11573 raft_consensus.cc:2781] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:22.873708 11573 raft_consensus.cc:482] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:22.873733 11573 raft_consensus.cc:3037] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.874751 11573 raft_consensus.cc:504] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.875116 11573 leader_election.cc:288] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 election: Requested vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:22.875527 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f338e79c67d409988cffea9a3c781de" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639"
I20260807 00:02:22.875582 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6f338e79c67d409988cffea9a3c781de" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270"
I20260807 00:02:22.875663 10959 raft_consensus.cc:3037] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.875738 11439 raft_consensus.cc:3037] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:22.876725 10959 raft_consensus.cc:2454] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 1.
I20260807 00:02:22.876746 11439 raft_consensus.cc:2454] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 1.
I20260807 00:02:22.877070 11078 leader_election.cc:302] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 78fb8157baad4f838c27aae825c67270, 9974f333337e4a399206e6ea902748ee, b625a2cfbb47489fa03310583e7a0639; no voters:
I20260807 00:02:22.877264 11573 raft_consensus.cc:2781] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:22.877357 11573 raft_consensus.cc:686] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [term 1 LEADER]: Becoming Leader. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Running, Role: LEADER
I20260807 00:02:22.877501 11573 consensus_queue.cc:237] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:22.878172 11594 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: TxnStatusTablet state changed. Reason: New leader 9974f333337e4a399206e6ea902748ee. Latest consensus state: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.878281 11594 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:02:22.878542 11646 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260807 00:02:22.878716 11646 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260807 00:02:22.879448 10721 catalog_manager.cc:5495] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 9974f333337e4a399206e6ea902748ee (127.30.204.66). New cstate: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } health_report { overall_health: UNKNOWN } } }
I20260807 00:02:22.896234 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.892535 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43968 (local address 127.30.204.66:34053)
0807 00:02:22.892662 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.892666 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.892754 (+ 88us) server_negotiation.cc:407] Connection header received
0807 00:02:22.892976 (+ 222us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.892979 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.893031 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.893109 (+ 78us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:22.893398 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.893954 (+ 556us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.895154 (+ 1200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.895375 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.895510 (+ 135us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.895821 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.895899 (+ 78us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.895947 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.896057 (+ 110us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.896111 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:02:22.896309 11632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.892426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:34053 (local address 127.0.0.1:43968)
0807 00:02:22.892685 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.892700 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.892821 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.893150 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.893153 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.893161 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:02:22.893277 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.893283 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.894055 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.894058 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.895020 (+ 962us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.895031 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.895517 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.895520 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.895618 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.895686 (+ 68us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.895970 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.895983 (+ 13us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.896051 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.896157 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152}
I20260807 00:02:22.898947 11439 raft_consensus.cc:1261] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Refusing update from remote peer 9974f333337e4a399206e6ea902748ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:22.898947 10959 raft_consensus.cc:1261] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Refusing update from remote peer 9974f333337e4a399206e6ea902748ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:22.899413 11594 consensus_queue.cc:1035] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:22.899621 11573 consensus_queue.cc:1035] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:22.901261 11575 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: TxnStatusTablet state changed. Reason: New leader 9974f333337e4a399206e6ea902748ee. Latest consensus state: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.901389 11575 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:02:22.901330 11572 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: TxnStatusTablet state changed. Reason: New leader 9974f333337e4a399206e6ea902748ee. Latest consensus state: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.901450 11572 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:02:22.903162 11594 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.903286 11594 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:02:22.904132 11591 mvcc.cc:205] Tried to move back new op lower bound from 7315705622109671424 to 7315705622027710464. Current Snapshot: MvccSnapshot[applied={T|T < 7315705622109671424}]
I20260807 00:02:22.904318 11572 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.904422 11572 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P 78fb8157baad4f838c27aae825c67270: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:02:22.907879 11578 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.908015 11578 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P 9974f333337e4a399206e6ea902748ee: This TxnStatusTablet replica's current role is: LEADER
I20260807 00:02:22.909124 11575 tablet_replica.cc:439] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } } }
I20260807 00:02:22.909245 11575 tablet_replica.cc:442] T 6f338e79c67d409988cffea9a3c781de P b625a2cfbb47489fa03310583e7a0639: This TxnStatusTablet replica's current role is: FOLLOWER
I20260807 00:02:22.912456 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.907935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42084 (local address 127.30.204.67:40065)
0807 00:02:22.908252 (+ 317us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.908261 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.908276 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:22.908373 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.908376 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.908446 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.908974 (+ 528us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:22.909101 (+ 127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.909819 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.910858 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.911148 (+ 290us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.911242 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.911553 (+ 311us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.912041 (+ 488us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.912150 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.912224 (+ 74us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.912286 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":252,"thread_start_us":137,"threads_started":1}
I20260807 00:02:22.924078 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.920661 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:34081 (local address 127.30.204.66:34053)
0807 00:02:22.920799 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.920805 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.921050 (+ 245us) server_negotiation.cc:407] Connection header received
0807 00:02:22.921189 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.921192 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.921244 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.921306 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:22.921563 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.922076 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.922983 (+ 907us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.923206 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.923295 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.923631 (+ 336us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.923712 (+ 81us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.923780 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.923869 (+ 89us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.923923 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:02:22.924548 11663 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.920545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:34053 (local address 127.30.204.67:34081)
0807 00:02:22.920965 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.920977 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.921091 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.921325 (+ 234us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.921328 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.921336 (+ 8us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:02:22.921465 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.921470 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.922181 (+ 711us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.922185 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.922866 (+ 681us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.922876 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.923324 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.923328 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.923418 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.923499 (+ 81us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.923794 (+ 295us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.923801 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.923859 (+ 58us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.923942 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":116,"threads_started":1}
I20260807 00:02:22.931533 10959 raft_consensus.cc:1261] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Refusing update from remote peer 78fb8157baad4f838c27aae825c67270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:22.931828 11191 raft_consensus.cc:1261] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Refusing update from remote peer 78fb8157baad4f838c27aae825c67270: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:22.932408 11622 consensus_queue.cc:1035] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:22.932591 11572 consensus_queue.cc:1035] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:22.957020 11670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.952972 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40065 (local address 127.30.204.66:35413)
0807 00:02:22.953323 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:22.953336 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:22.953442 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:22.953807 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:22.953810 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:22.953819 (+ 9us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:02:22.953910 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.953915 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.954850 (+ 935us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.954854 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.955715 (+ 861us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:22.955727 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.956313 (+ 586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.956316 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:22.956379 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.956432 (+ 53us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.956716 (+ 284us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.956721 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:02:22.956778 (+ 57us) client_negotiation.cc:240] Negotiation successful
0807 00:02:22.956865 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":124,"threads_started":1}
I20260807 00:02:22.957010 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:22.953050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:35413 (local address 127.30.204.67:40065)
0807 00:02:22.953172 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:22.953177 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:22.953405 (+ 228us) server_negotiation.cc:407] Connection header received
0807 00:02:22.953527 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:22.953530 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:22.953606 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:22.953672 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:22.954018 (+ 346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.954724 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.955840 (+ 1116us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:22.956208 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:22.956321 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:22.956552 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:22.956645 (+ 93us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:22.956712 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:22.956796 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 00:02:22.956858 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 00:02:22.988837 10720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:22.989037 10720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:43410:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:22.991677 10720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Sending DeleteTablet for 3 replicas of tablet 323c3bc0d2634775b26644f393a3f2d2
I20260807 00:02:22.992408 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet 323c3bc0d2634775b26644f393a3f2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:22 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58426
I20260807 00:02:22.992823 11150 tablet_service.cc:1514] Processing DeleteTablet for tablet 323c3bc0d2634775b26644f393a3f2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:22 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:43944
I20260807 00:02:22.992893 11675 tablet_replica.cc:330] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: stopping tablet replica
I20260807 00:02:22.993021 11605 tablet_replica.cc:330] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: stopping tablet replica
I20260807 00:02:22.993147 11605 raft_consensus.cc:2229] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:22.993165 11675 raft_consensus.cc:2229] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:22.993292 11605 raft_consensus.cc:2258] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:22.993412 11675 raft_consensus.cc:2258] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:22.994153 11605 ts_tablet_manager.cc:1905] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:22.994207 11675 ts_tablet_manager.cc:1905] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:22.997231 11605 ts_tablet_manager.cc:1918] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:02:22.997326 11605 log.cc:1196] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/323c3bc0d2634775b26644f393a3f2d2
I20260807 00:02:22.997581 11605 ts_tablet_manager.cc:1939] T 323c3bc0d2634775b26644f393a3f2d2 P 9974f333337e4a399206e6ea902748ee: Deleting consensus metadata
I20260807 00:02:22.997714 11675 ts_tablet_manager.cc:1918] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:02:22.997861 11675 log.cc:1196] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/323c3bc0d2634775b26644f393a3f2d2
I20260807 00:02:22.998191 11675 ts_tablet_manager.cc:1939] T 323c3bc0d2634775b26644f393a3f2d2 P b625a2cfbb47489fa03310583e7a0639: Deleting consensus metadata
I20260807 00:02:22.998420 10703 catalog_manager.cc:4841] TS 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053): tablet 323c3bc0d2634775b26644f393a3f2d2 (table test-table [id=cf422f7bb666435aa08b417ab62892bc]) successfully deleted
I20260807 00:02:22.999109 10704 catalog_manager.cc:4841] TS b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655): tablet 323c3bc0d2634775b26644f393a3f2d2 (table test-table [id=cf422f7bb666435aa08b417ab62892bc]) successfully deleted
I20260807 00:02:22.999701 11405 tablet_service.cc:1514] Processing DeleteTablet for tablet 323c3bc0d2634775b26644f393a3f2d2 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:22 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:42070
I20260807 00:02:23.008472 11678 tablet_replica.cc:330] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: stopping tablet replica
I20260807 00:02:23.008649 11678 raft_consensus.cc:2229] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:23.009200 11678 raft_consensus.cc:2258] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:23.010372 11678 ts_tablet_manager.cc:1905] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:23.013856 11678 ts_tablet_manager.cc:1918] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:02:23.013955 11678 log.cc:1196] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/323c3bc0d2634775b26644f393a3f2d2
I20260807 00:02:23.014245 11678 ts_tablet_manager.cc:1939] T 323c3bc0d2634775b26644f393a3f2d2 P 78fb8157baad4f838c27aae825c67270: Deleting consensus metadata
I20260807 00:02:23.015125 10701 catalog_manager.cc:4841] TS 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065): tablet 323c3bc0d2634775b26644f393a3f2d2 (table test-table [id=cf422f7bb666435aa08b417ab62892bc]) successfully deleted
Aug 07 00:02:23 dist-test-slave-pww0 krb5kdc[10650](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Aug 07 00:02:23 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:23 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:02:23.041375 11633 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:23.031998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43432 (local address 127.30.204.126:34527)
0807 00:02:23.032147 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:23.032152 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:23.032841 (+ 689us) server_negotiation.cc:407] Connection header received
0807 00:02:23.033041 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:23.033045 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:23.033100 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:23.033156 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:23.034188 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.035071 (+ 883us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.036748 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.037132 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.037275 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.041219 (+ 3944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43432: BlockingRecv error: recv got EOF from 127.0.0.1:43432 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
Aug 07 00:02:23 dist-test-slave-pww0 krb5kdc[10650](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060943, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM
I20260807 00:02:23.054203 11633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.043174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43446 (local address 127.30.204.126:34527)
0807 00:02:23.043347 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:23.043352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:23.043776 (+ 424us) server_negotiation.cc:407] Connection header received
0807 00:02:23.044045 (+ 269us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:23.044049 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:23.044115 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:23.044205 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:23.045000 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.045873 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.047042 (+ 1169us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.047400 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.047577 (+ 177us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.050598 (+ 3021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:23.050602 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:23.050605 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:23.050643 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:23.052327 (+ 1684us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:23.052876 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:23.052880 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:23.052882 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:23.052991 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:23.053290 (+ 299us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:23.053293 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:23.053295 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:23.053547 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:23.053655 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:23.053905 (+ 250us) server_negotiation.cc:299] Negotiation successful
0807 00:02:23.054027 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:02:23.057684 10720 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:23.058187 10720 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:23.063439 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 53d2ec066577434ab03d88f5c796f1b8 (DEFAULT_TABLE table=test-table [id=4c15666c728a4c1a80e50df2e5f83ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:23.063714 11150 tablet_service.cc:1467] Processing CreateTablet for tablet 53d2ec066577434ab03d88f5c796f1b8 (DEFAULT_TABLE table=test-table [id=4c15666c728a4c1a80e50df2e5f83ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:23.063850 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d2ec066577434ab03d88f5c796f1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:23.063880 11405 tablet_service.cc:1467] Processing CreateTablet for tablet 53d2ec066577434ab03d88f5c796f1b8 (DEFAULT_TABLE table=test-table [id=4c15666c728a4c1a80e50df2e5f83ae3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:23.064023 11150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d2ec066577434ab03d88f5c796f1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:23.064131 11405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 53d2ec066577434ab03d88f5c796f1b8. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:23.066455 11563 tablet_bootstrap.cc:492] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: Bootstrap starting.
I20260807 00:02:23.066623 11564 tablet_bootstrap.cc:492] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: Bootstrap starting.
I20260807 00:02:23.067519 11564 tablet_bootstrap.cc:654] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:23.067551 11563 tablet_bootstrap.cc:654] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:23.068168 11562 tablet_bootstrap.cc:492] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: Bootstrap starting.
I20260807 00:02:23.068985 11562 tablet_bootstrap.cc:654] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:23.069135 11563 tablet_bootstrap.cc:492] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: No bootstrap required, opened a new log
I20260807 00:02:23.069233 11563 ts_tablet_manager.cc:1397] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260807 00:02:23.069711 11563 raft_consensus.cc:348] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.069844 11563 raft_consensus.cc:374] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:23.069885 11563 raft_consensus.cc:729] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b625a2cfbb47489fa03310583e7a0639, State: Initialized, Role: FOLLOWER
I20260807 00:02:23.070014 11563 consensus_queue.cc:260] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.070312 11563 ts_tablet_manager.cc:1428] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260807 00:02:23.070935 11562 tablet_bootstrap.cc:492] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: No bootstrap required, opened a new log
I20260807 00:02:23.071031 11562 ts_tablet_manager.cc:1397] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:23.071494 11562 raft_consensus.cc:348] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.071599 11562 raft_consensus.cc:374] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:23.071619 11562 raft_consensus.cc:729] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Initialized, Role: FOLLOWER
I20260807 00:02:23.071735 11562 consensus_queue.cc:260] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.072008 11562 ts_tablet_manager.cc:1428] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:23.072677 11564 tablet_bootstrap.cc:492] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: No bootstrap required, opened a new log
I20260807 00:02:23.072781 11564 ts_tablet_manager.cc:1397] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260807 00:02:23.073177 11564 raft_consensus.cc:348] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.073266 11564 raft_consensus.cc:374] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:23.073285 11564 raft_consensus.cc:729] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb8157baad4f838c27aae825c67270, State: Initialized, Role: FOLLOWER
I20260807 00:02:23.073390 11564 consensus_queue.cc:260] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.073662 11564 ts_tablet_manager.cc:1428] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:23.129596 11575 raft_consensus.cc:482] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:23.129767 11575 raft_consensus.cc:504] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.131285 11575 leader_election.cc:288] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:23.136551 11656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.131709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:59351 (local address 127.30.204.67:40065)
0807 00:02:23.131945 (+ 236us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:23.131951 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:23.132197 (+ 246us) server_negotiation.cc:407] Connection header received
0807 00:02:23.132378 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:23.132382 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:23.132429 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:23.132491 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:23.133096 (+ 605us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.133833 (+ 737us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.135165 (+ 1332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.135891 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.136027 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.136098 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:23.136318 (+ 220us) server_negotiation.cc:299] Negotiation successful
0807 00:02:23.136405 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150}
I20260807 00:02:23.136597 11709 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.131640 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40065 (local address 127.30.204.65:59351)
0807 00:02:23.132064 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:23.132079 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:23.132252 (+ 173us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:23.132513 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:23.132515 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:23.132531 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:23.132959 (+ 428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:23.132967 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.133986 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.133989 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:23.134974 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:23.134986 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.136056 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.136061 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:23.136171 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.136201 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 00:02:23.136297 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 00:02:23.136387 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":36,"thread_start_us":81,"threads_started":1}
I20260807 00:02:23.136811 11554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.132179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:60069 (local address 127.30.204.66:34053)
0807 00:02:23.132300 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:23.132305 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:23.132320 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:23.132378 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:23.132381 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:23.132438 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:23.132526 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:23.132972 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.133789 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.135484 (+ 1695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.136074 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.136187 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.136233 (+ 46us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:23.136601 (+ 368us) server_negotiation.cc:299] Negotiation successful
0807 00:02:23.136693 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:02:23.137368 11708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.131706 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:34053 (local address 127.30.204.65:60069)
0807 00:02:23.132108 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:23.132119 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:23.132238 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:23.132679 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:23.132682 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:23.132704 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:23.132823 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:23.132831 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.133929 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.133933 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:23.135309 (+ 1376us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:23.135323 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.136295 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.136298 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:23.136398 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.136423 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:02:23.137138 (+ 715us) client_negotiation.cc:240] Negotiation successful
0807 00:02:23.137220 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":342,"mutex_wait_us":73,"thread_start_us":166,"threads_started":1}
I20260807 00:02:23.137305 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d2ec066577434ab03d88f5c796f1b8" candidate_uuid: "b625a2cfbb47489fa03310583e7a0639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270" is_pre_election: true
I20260807 00:02:23.137476 11439 raft_consensus.cc:2454] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b625a2cfbb47489fa03310583e7a0639 in term 0.
I20260807 00:02:23.137835 11191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d2ec066577434ab03d88f5c796f1b8" candidate_uuid: "b625a2cfbb47489fa03310583e7a0639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9974f333337e4a399206e6ea902748ee" is_pre_election: true
I20260807 00:02:23.137916 10855 leader_election.cc:302] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb8157baad4f838c27aae825c67270, b625a2cfbb47489fa03310583e7a0639; no voters:
I20260807 00:02:23.137995 11191 raft_consensus.cc:2454] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b625a2cfbb47489fa03310583e7a0639 in term 0.
I20260807 00:02:23.138240 11575 raft_consensus.cc:2781] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:23.138305 11575 raft_consensus.cc:482] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:23.138339 11575 raft_consensus.cc:3037] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:23.139689 11575 raft_consensus.cc:504] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.140085 11575 leader_election.cc:288] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [CANDIDATE]: Term 1 election: Requested vote from peers 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:23.140439 11191 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d2ec066577434ab03d88f5c796f1b8" candidate_uuid: "b625a2cfbb47489fa03310583e7a0639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "9974f333337e4a399206e6ea902748ee"
I20260807 00:02:23.140571 11191 raft_consensus.cc:3037] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:23.140552 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "53d2ec066577434ab03d88f5c796f1b8" candidate_uuid: "b625a2cfbb47489fa03310583e7a0639" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270"
I20260807 00:02:23.140693 11439 raft_consensus.cc:3037] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:23.141819 11191 raft_consensus.cc:2454] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b625a2cfbb47489fa03310583e7a0639 in term 1.
I20260807 00:02:23.142241 11439 raft_consensus.cc:2454] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b625a2cfbb47489fa03310583e7a0639 in term 1.
I20260807 00:02:23.142267 10856 leader_election.cc:302] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 9974f333337e4a399206e6ea902748ee, b625a2cfbb47489fa03310583e7a0639; no voters:
I20260807 00:02:23.142513 11575 raft_consensus.cc:2781] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:23.142819 11575 raft_consensus.cc:686] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 1 LEADER]: Becoming Leader. State: Replica: b625a2cfbb47489fa03310583e7a0639, State: Running, Role: LEADER
I20260807 00:02:23.143095 11575 consensus_queue.cc:237] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.145969 10721 catalog_manager.cc:5495] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 reported cstate change: term changed from 0 to 1, leader changed from <none> to b625a2cfbb47489fa03310583e7a0639 (127.30.204.65). New cstate: current_term: 1 leader_uuid: "b625a2cfbb47489fa03310583e7a0639" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } health_report { overall_health: UNKNOWN } } }
I20260807 00:02:23.166785 11559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.163119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58430 (local address 127.30.204.65:44655)
0807 00:02:23.163272 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:23.163276 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:23.163292 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:23.163387 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:23.163390 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:23.163450 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:23.163596 (+ 146us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:23.163844 (+ 248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.164367 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.165234 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.165470 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.165564 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.165866 (+ 302us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:23.166369 (+ 503us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:23.166482 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:23.166555 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 00:02:23.166629 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:02:23.169732 11439 raft_consensus.cc:1261] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Refusing update from remote peer b625a2cfbb47489fa03310583e7a0639: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:23.169750 11191 raft_consensus.cc:1261] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Refusing update from remote peer b625a2cfbb47489fa03310583e7a0639: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:23.170248 11711 consensus_queue.cc:1035] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:23.170447 11575 consensus_queue.cc:1035] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [LEADER]: Connected to new peer: Peer: permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:23.182322 10721 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:23.182550 10721 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:23.184975 10721 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Sending DeleteTablet for 3 replicas of tablet 53d2ec066577434ab03d88f5c796f1b8
I20260807 00:02:23.185689 11150 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d2ec066577434ab03d88f5c796f1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:23 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:43944
I20260807 00:02:23.185698 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d2ec066577434ab03d88f5c796f1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:23 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58426
I20260807 00:02:23.185927 11675 tablet_replica.cc:330] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: stopping tablet replica
I20260807 00:02:23.185937 11405 tablet_service.cc:1514] Processing DeleteTablet for tablet 53d2ec066577434ab03d88f5c796f1b8 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:23 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:42070
I20260807 00:02:23.186051 11675 raft_consensus.cc:2229] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:23.186187 11605 tablet_replica.cc:330] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: stopping tablet replica
I20260807 00:02:23.186316 11605 raft_consensus.cc:2229] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:23.186414 11675 raft_consensus.cc:2258] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:23.186465 11605 raft_consensus.cc:2258] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:23.186653 11678 tablet_replica.cc:330] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: stopping tablet replica
I20260807 00:02:23.186759 11678 raft_consensus.cc:2229] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:23.186858 11678 raft_consensus.cc:2258] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:23.186792 10721 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43446:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:23.187269 10721 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:23.187736 11678 ts_tablet_manager.cc:1905] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:23.188125 11675 ts_tablet_manager.cc:1905] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:23.188287 11605 ts_tablet_manager.cc:1905] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:23.190222 11605 ts_tablet_manager.cc:1918] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:23.190303 11605 log.cc:1196] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/53d2ec066577434ab03d88f5c796f1b8
I20260807 00:02:23.190487 11605 ts_tablet_manager.cc:1939] T 53d2ec066577434ab03d88f5c796f1b8 P 9974f333337e4a399206e6ea902748ee: Deleting consensus metadata
I20260807 00:02:23.190515 11678 ts_tablet_manager.cc:1918] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:23.190593 11678 log.cc:1196] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/53d2ec066577434ab03d88f5c796f1b8
I20260807 00:02:23.190572 11675 ts_tablet_manager.cc:1918] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:23.190636 11675 log.cc:1196] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/53d2ec066577434ab03d88f5c796f1b8
I20260807 00:02:23.190855 11678 ts_tablet_manager.cc:1939] T 53d2ec066577434ab03d88f5c796f1b8 P 78fb8157baad4f838c27aae825c67270: Deleting consensus metadata
I20260807 00:02:23.190899 11675 ts_tablet_manager.cc:1939] T 53d2ec066577434ab03d88f5c796f1b8 P b625a2cfbb47489fa03310583e7a0639: Deleting consensus metadata
I20260807 00:02:23.191597 10701 catalog_manager.cc:4841] TS 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065): tablet 53d2ec066577434ab03d88f5c796f1b8 (table test-table [id=4c15666c728a4c1a80e50df2e5f83ae3]) successfully deleted
I20260807 00:02:23.191597 10703 catalog_manager.cc:4841] TS 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053): tablet 53d2ec066577434ab03d88f5c796f1b8 (table test-table [id=4c15666c728a4c1a80e50df2e5f83ae3]) successfully deleted
I20260807 00:02:23.192556 10704 catalog_manager.cc:4841] TS b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655): tablet 53d2ec066577434ab03d88f5c796f1b8 (table test-table [id=4c15666c728a4c1a80e50df2e5f83ae3]) successfully deleted
I20260807 00:02:23.195947 10932 tablet_service.cc:1467] Processing CreateTablet for tablet 8985acd9ca304fc781bc74b5375dfc85 (DEFAULT_TABLE table=test-table [id=9cab5f5eaf0c455294a938d6272cdfb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:23.195956 11150 tablet_service.cc:1467] Processing CreateTablet for tablet 8985acd9ca304fc781bc74b5375dfc85 (DEFAULT_TABLE table=test-table [id=9cab5f5eaf0c455294a938d6272cdfb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:23.196285 10932 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8985acd9ca304fc781bc74b5375dfc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:23.196285 11150 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8985acd9ca304fc781bc74b5375dfc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:23.198764 11562 tablet_bootstrap.cc:492] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: Bootstrap starting.
I20260807 00:02:23.199222 11405 tablet_service.cc:1467] Processing CreateTablet for tablet 8985acd9ca304fc781bc74b5375dfc85 (DEFAULT_TABLE table=test-table [id=9cab5f5eaf0c455294a938d6272cdfb3]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:23.199652 11405 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8985acd9ca304fc781bc74b5375dfc85. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:23.199681 11563 tablet_bootstrap.cc:492] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639: Bootstrap starting.
I20260807 00:02:23.199695 11562 tablet_bootstrap.cc:654] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:23.200567 11563 tablet_bootstrap.cc:654] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:23.201377 11562 tablet_bootstrap.cc:492] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: No bootstrap required, opened a new log
I20260807 00:02:23.201463 11562 ts_tablet_manager.cc:1397] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:23.201874 11562 raft_consensus.cc:348] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.201982 11562 raft_consensus.cc:374] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:23.202009 11562 raft_consensus.cc:729] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Initialized, Role: FOLLOWER
I20260807 00:02:23.202044 11564 tablet_bootstrap.cc:492] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: Bootstrap starting.
I20260807 00:02:23.202147 11562 consensus_queue.cc:260] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.202401 11562 ts_tablet_manager.cc:1428] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:23.202404 11563 tablet_bootstrap.cc:492] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639: No bootstrap required, opened a new log
I20260807 00:02:23.202483 11563 ts_tablet_manager.cc:1397] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260807 00:02:23.202870 11564 tablet_bootstrap.cc:654] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:23.202968 11563 raft_consensus.cc:348] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.203107 11563 raft_consensus.cc:374] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:23.203141 11563 raft_consensus.cc:729] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b625a2cfbb47489fa03310583e7a0639, State: Initialized, Role: FOLLOWER
I20260807 00:02:23.203269 11563 consensus_queue.cc:260] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.203553 11563 ts_tablet_manager.cc:1428] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:23.204414 11564 tablet_bootstrap.cc:492] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: No bootstrap required, opened a new log
I20260807 00:02:23.204510 11564 ts_tablet_manager.cc:1397] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:23.204929 11564 raft_consensus.cc:348] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.205041 11564 raft_consensus.cc:374] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:23.205071 11564 raft_consensus.cc:729] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 78fb8157baad4f838c27aae825c67270, State: Initialized, Role: FOLLOWER
I20260807 00:02:23.205194 11564 consensus_queue.cc:260] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.205451 11564 ts_tablet_manager.cc:1428] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:02:23.593202 11573 raft_consensus.cc:482] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:23.593375 11573 raft_consensus.cc:504] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.593788 11573 leader_election.cc:288] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:23.594362 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8985acd9ca304fc781bc74b5375dfc85" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639" is_pre_election: true
I20260807 00:02:23.594362 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8985acd9ca304fc781bc74b5375dfc85" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270" is_pre_election: true
I20260807 00:02:23.594535 10959 raft_consensus.cc:2454] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 0.
I20260807 00:02:23.594535 11439 raft_consensus.cc:2454] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 0.
I20260807 00:02:23.594918 11078 leader_election.cc:302] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 78fb8157baad4f838c27aae825c67270, 9974f333337e4a399206e6ea902748ee, b625a2cfbb47489fa03310583e7a0639; no voters:
I20260807 00:02:23.595135 11573 raft_consensus.cc:2781] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:23.595201 11573 raft_consensus.cc:482] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:23.595230 11573 raft_consensus.cc:3037] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:23.596287 11573 raft_consensus.cc:504] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.596676 11573 leader_election.cc:288] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 election: Requested vote from peers b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655), 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065)
I20260807 00:02:23.597106 11439 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8985acd9ca304fc781bc74b5375dfc85" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "78fb8157baad4f838c27aae825c67270"
I20260807 00:02:23.597105 10959 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8985acd9ca304fc781bc74b5375dfc85" candidate_uuid: "9974f333337e4a399206e6ea902748ee" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b625a2cfbb47489fa03310583e7a0639"
I20260807 00:02:23.597254 10959 raft_consensus.cc:3037] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:23.597254 11439 raft_consensus.cc:3037] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:23.598415 10959 raft_consensus.cc:2454] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 1.
I20260807 00:02:23.598415 11439 raft_consensus.cc:2454] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 9974f333337e4a399206e6ea902748ee in term 1.
I20260807 00:02:23.598755 11075 leader_election.cc:302] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 78fb8157baad4f838c27aae825c67270, 9974f333337e4a399206e6ea902748ee; no voters:
I20260807 00:02:23.598979 11573 raft_consensus.cc:2781] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:23.599155 11573 raft_consensus.cc:686] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 1 LEADER]: Becoming Leader. State: Replica: 9974f333337e4a399206e6ea902748ee, State: Running, Role: LEADER
I20260807 00:02:23.599300 11573 consensus_queue.cc:237] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } }
I20260807 00:02:23.601073 10720 catalog_manager.cc:5495] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee reported cstate change: term changed from 0 to 1, leader changed from <none> to 9974f333337e4a399206e6ea902748ee (127.30.204.66). New cstate: current_term: 1 leader_uuid: "9974f333337e4a399206e6ea902748ee" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "9974f333337e4a399206e6ea902748ee" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 34053 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 } health_report { overall_health: UNKNOWN } } }
I20260807 00:02:23.669500 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.666580 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43974 (local address 127.30.204.66:34053)
0807 00:02:23.666793 (+ 213us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:23.666796 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:23.666808 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:02:23.666847 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:23.666849 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:23.666890 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:23.666962 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:23.667185 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.667776 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.668616 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.668848 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.668919 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.669115 (+ 196us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:23.669187 (+ 72us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:23.669236 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:23.669296 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 00:02:23.669340 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":92,"threads_started":1}
I20260807 00:02:23.678645 11741 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.675501 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:34053 (local address 127.30.204.66:38493)
0807 00:02:23.675778 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:23.675793 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:23.675888 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:23.676089 (+ 201us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:23.676091 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:23.676097 (+ 6us) client_negotiation.cc:189] Negotiated authn=TOKEN
0807 00:02:23.676186 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:23.676190 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.676970 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.676973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:23.677595 (+ 622us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:23.677604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.677986 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.677988 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:23.678051 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.678077 (+ 26us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0807 00:02:23.678350 (+ 273us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0807 00:02:23.678356 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:02:23.678410 (+ 54us) client_negotiation.cc:240] Negotiation successful
0807 00:02:23.678516 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":91,"threads_started":1}
I20260807 00:02:23.678645 11735 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.675675 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:38493 (local address 127.30.204.66:34053)
0807 00:02:23.675798 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:23.675801 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:23.675853 (+ 52us) server_negotiation.cc:407] Connection header received
0807 00:02:23.675978 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:23.675981 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:23.676023 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:23.676077 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:23.676261 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.676852 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.677693 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:23.677910 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:23.677976 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:23.678208 (+ 232us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:23.678278 (+ 70us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:23.678339 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:23.678426 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 00:02:23.678473 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:02:23.684784 11439 raft_consensus.cc:1261] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Refusing update from remote peer 9974f333337e4a399206e6ea902748ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:23.684814 10959 raft_consensus.cc:1261] T 8985acd9ca304fc781bc74b5375dfc85 P b625a2cfbb47489fa03310583e7a0639 [term 1 FOLLOWER]: Refusing update from remote peer 9974f333337e4a399206e6ea902748ee: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:23.685170 11733 consensus_queue.cc:1035] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "b625a2cfbb47489fa03310583e7a0639" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 44655 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:23.685364 11573 consensus_queue.cc:1035] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [LEADER]: Connected to new peer: Peer: permanent_uuid: "78fb8157baad4f838c27aae825c67270" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40065 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:23.718286 10720 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:23.718472 10720 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43446:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:23.720728 10720 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 27484d8abb864e2da97fd6353f869cb0: Sending DeleteTablet for 3 replicas of tablet 8985acd9ca304fc781bc74b5375dfc85
I20260807 00:02:23.721369 10932 tablet_service.cc:1514] Processing DeleteTablet for tablet 8985acd9ca304fc781bc74b5375dfc85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:23 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58426
I20260807 00:02:23.721361 11148 tablet_service.cc:1514] Processing DeleteTablet for tablet 8985acd9ca304fc781bc74b5375dfc85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:23 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:43944
I20260807 00:02:23.721477 11405 tablet_service.cc:1514] Processing DeleteTablet for tablet 8985acd9ca304fc781bc74b5375dfc85 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:23 UTC) from {username='oryx', principal='oryx/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:42070
I20260807 00:02:23.721760 11750 tablet_replica.cc:330] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: stopping tablet replica
I20260807 00:02:23.721801 11751 tablet_replica.cc:330] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: stopping tablet replica
I20260807 00:02:23.721904 11751 raft_consensus.cc:2229] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:23.721904 11750 raft_consensus.cc:2229] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:23.722025 11751 raft_consensus.cc:2258] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:23.722115 11750 raft_consensus.cc:2258] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:23.722276 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 10827
I20260807 00:02:23.730031 11751 ts_tablet_manager.cc:1905] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:23.733037 11751 ts_tablet_manager.cc:1918] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:02:23.733145 11751 log.cc:1196] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/8985acd9ca304fc781bc74b5375dfc85
I20260807 00:02:23.733355 11751 ts_tablet_manager.cc:1939] T 8985acd9ca304fc781bc74b5375dfc85 P 78fb8157baad4f838c27aae825c67270: Deleting consensus metadata
I20260807 00:02:23.734215 10701 catalog_manager.cc:4841] TS 78fb8157baad4f838c27aae825c67270 (127.30.204.67:40065): tablet 8985acd9ca304fc781bc74b5375dfc85 (table test-table [id=9cab5f5eaf0c455294a938d6272cdfb3]) successfully deleted
I20260807 00:02:23.735436 11750 ts_tablet_manager.cc:1905] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:23.737759 11750 ts_tablet_manager.cc:1918] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260807 00:02:23.737839 11750 log.cc:1196] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/8985acd9ca304fc781bc74b5375dfc85
I20260807 00:02:23.738083 11750 ts_tablet_manager.cc:1939] T 8985acd9ca304fc781bc74b5375dfc85 P 9974f333337e4a399206e6ea902748ee: Deleting consensus metadata
I20260807 00:02:23.738879 10703 catalog_manager.cc:4841] TS 9974f333337e4a399206e6ea902748ee (127.30.204.66:34053): tablet 8985acd9ca304fc781bc74b5375dfc85 (table test-table [id=9cab5f5eaf0c455294a938d6272cdfb3]) successfully deleted
W20260807 00:02:23.741865 10704 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:44655 (error 108)
I20260807 00:02:23.742753 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 11049
I20260807 00:02:23.749627 11756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:23.742621 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:44655 (local address 127.0.0.1:58434)
0807 00:02:23.749355 (+ 6734us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:23.749494 (+ 139us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:44655: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":60,"threads_started":1}
W20260807 00:02:23.749903 10704 catalog_manager.cc:4584] TS b625a2cfbb47489fa03310583e7a0639 (127.30.204.65:44655): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8985acd9ca304fc781bc74b5375dfc85: Network error: Client connection negotiation failed: client connection to 127.30.204.65:44655: connect: Connection refused (error 111)
I20260807 00:02:23.761927 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 11290
I20260807 00:02:23.780123 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 10671
2026-08-07T00:02:23Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (4101 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:23 dist-test-slave-pww0 krb5kdc[11761](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:23 dist-test-slave-pww0 krb5kdc[11761](info): set up 2 sockets
Aug 07 00:02:23 dist-test-slave-pww0 krb5kdc[11761](info): commencing operation
krb5kdc: starting...
W20260807 00:02:24.951793 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:25 dist-test-slave-pww0 krb5kdc[11761](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:25Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:25Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:25.090641 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:38941
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36321
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:38941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:25.214733 11784 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:25.215085 11784 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:25.215147 11784 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:25.219493 11784 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:25.219591 11784 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:25.219619 11784 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:25.219640 11784 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:25.219671 11784 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:25.225600 11784 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36321
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:38941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:38941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11784
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:25.226871 11784 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:25.228091 11784 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:25.237697 11798 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:25.237838 11784 server_base.cc:1034] running on GCE node
W20260807 00:02:25.237752 11794 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:25.238032 11793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:25.238709 11784 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:25.240075 11784 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:25.241271 11784 hybrid_clock.cc:648] HybridClock initialized: now 1786060945241253 us; error 35 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-08-07T00:02:25Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1461 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:25 dist-test-slave-pww0 krb5kdc[11804](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:25 dist-test-slave-pww0 krb5kdc[11804](info): set up 2 sockets
Aug 07 00:02:25 dist-test-slave-pww0 krb5kdc[11804](info): commencing operation
krb5kdc: starting...
W20260807 00:02:26.429661 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.156s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:26 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:26.566310 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43273
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:26.686393 11825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:26.686719 11825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:26.686784 11825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:26.691437 11825 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:26.691546 11825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:26.691573 11825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:26.691596 11825 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:26.691624 11825 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:26.697422 11825 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:26.698652 11825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:26.699853 11825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:26.708711 11835 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:26.708736 11831 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:26.708863 11825 server_base.cc:1034] running on GCE node
W20260807 00:02:26.708714 11832 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:26.709678 11825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:26.710925 11825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:26.712113 11825 hybrid_clock.cc:648] HybridClock initialized: now 1786060946712091 us; error 35 us; skew 500 ppm
Aug 07 00:02:26 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060946, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:26.718649 11825 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:26.719864 11825 webserver.cc:460] Webserver started at http://127.30.204.126:38101/ using document root <none> and password file <none>
I20260807 00:02:26.720464 11825 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:26.720538 11825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:26.720780 11825 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:26.723061 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "198b6c0493024d229b5591347cf6fb4a"
format_stamp: "Formatted at 2026-08-07 00:02:26 on dist-test-slave-pww0"
server_key: "2de69045333ce7150ca56e207fd935ea"
server_key_iv: "65d8030ec4cb6d709702a35061051173"
server_key_version: "encryptionkey@0"
I20260807 00:02:26.723770 11825 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "198b6c0493024d229b5591347cf6fb4a"
format_stamp: "Formatted at 2026-08-07 00:02:26 on dist-test-slave-pww0"
server_key: "2de69045333ce7150ca56e207fd935ea"
server_key_iv: "65d8030ec4cb6d709702a35061051173"
server_key_version: "encryptionkey@0"
I20260807 00:02:26.728113 11825 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:26.730953 11845 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:26.732403 11825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:26.732549 11825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "198b6c0493024d229b5591347cf6fb4a"
format_stamp: "Formatted at 2026-08-07 00:02:26 on dist-test-slave-pww0"
server_key: "2de69045333ce7150ca56e207fd935ea"
server_key_iv: "65d8030ec4cb6d709702a35061051173"
server_key_version: "encryptionkey@0"
I20260807 00:02:26.732682 11825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:26.772722 11825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:26.776072 11825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:26.776275 11825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:26.783581 11825 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:43273
I20260807 00:02:26.783578 11932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:43273 every 8 connection(s)
I20260807 00:02:26.784679 11825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:26.784873 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 11825
I20260807 00:02:26.785008 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:26.785390 31537 external_mini_cluster.cc:1428] Setting key 07ccba6f1916cd3f268f440a55f31fc0
I20260807 00:02:26.788345 11933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
Aug 07 00:02:26 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:26.795498 11933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Bootstrap starting.
I20260807 00:02:26.798449 11933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:26.799379 11933 log.cc:824] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:26.801486 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:26.786892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38984 (local address 127.30.204.126:43273)
0807 00:02:26.787348 (+ 456us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:26.787368 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:26.787412 (+ 44us) server_negotiation.cc:407] Connection header received
0807 00:02:26.788306 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:26.788331 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:26.788789 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:26.789177 (+ 388us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:26.789946 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:26.791530 (+ 1584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:26.792511 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:26.793196 (+ 685us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:26.793356 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:26.796138 (+ 2782us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:26.796144 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:26.796161 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:26.796198 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:26.798161 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:26.798724 (+ 563us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:26.798731 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:26.798738 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:26.798901 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:26.799156 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:26.799159 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:26.799161 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:26.799754 (+ 593us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:26.799954 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:26.800183 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 00:02:26.800491 (+ 308us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":129,"threads_started":1}
I20260807 00:02:26.802287 11933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: No bootstrap required, opened a new log
I20260807 00:02:26.806594 11933 raft_consensus.cc:348] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:26.806922 11933 raft_consensus.cc:374] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:26.806990 11933 raft_consensus.cc:729] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198b6c0493024d229b5591347cf6fb4a, State: Initialized, Role: FOLLOWER
I20260807 00:02:26.807523 11933 consensus_queue.cc:260] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:26.807678 11933 raft_consensus.cc:388] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:26.807746 11933 raft_consensus.cc:482] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:26.807847 11933 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:26.808867 11933 raft_consensus.cc:504] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:26.809244 11933 leader_election.cc:302] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 198b6c0493024d229b5591347cf6fb4a; no voters:
I20260807 00:02:26.809576 11933 leader_election.cc:288] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:26.809741 11940 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:26.809989 11940 raft_consensus.cc:686] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 LEADER]: Becoming Leader. State: Replica: 198b6c0493024d229b5591347cf6fb4a, State: Running, Role: LEADER
I20260807 00:02:26.810329 11940 consensus_queue.cc:237] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:26.810926 11933 sys_catalog.cc:564] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:26.812028 11943 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 198b6c0493024d229b5591347cf6fb4a. Latest consensus state: current_term: 1 leader_uuid: "198b6c0493024d229b5591347cf6fb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } } }
I20260807 00:02:26.812168 11943 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:26.812561 11952 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:26.812492 11941 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "198b6c0493024d229b5591347cf6fb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } } }
I20260807 00:02:26.813431 11941 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:26.816596 11952 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:26.823978 11952 catalog_manager.cc:1347] Generated new cluster ID: 3c83c89e0ac2479eacc8df904cd083b7
I20260807 00:02:26.824077 11952 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:26.835901 11952 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:26.837353 11952 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:26.849417 11952 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Generated new TSK 0
I20260807 00:02:26.850445 11952 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:02:26.887141 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43273
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:27.014073 11968 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:27.014439 11968 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:27.014492 11968 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:27.018848 11968 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:27.018937 11968 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:27.019001 11968 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:27.024864 11968 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.11968
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:27.026113 11968 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:27.027264 11968 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:27.038056 11974 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:27.038061 11976 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:27.038060 11973 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:27.038291 11968 server_base.cc:1034] running on GCE node
I20260807 00:02:27.038826 11968 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:27.039623 11968 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:27.040839 11968 hybrid_clock.cc:648] HybridClock initialized: now 1786060947040812 us; error 38 us; skew 500 ppm
Aug 07 00:02:27 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:27.044106 11968 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:02:27.045285 11968 webserver.cc:460] Webserver started at http://127.30.204.65:39131/ using document root <none> and password file <none>
I20260807 00:02:27.046105 11968 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:27.046223 11968 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:27.046476 11968 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:27.048905 11968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "aea436054f314a08b209726d18174f89"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "cc93097d5e833df609a2539a2318d96b"
server_key_iv: "3a7109627b0f668c1c2022bcc22dbfb8"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.049562 11968 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "aea436054f314a08b209726d18174f89"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "cc93097d5e833df609a2539a2318d96b"
server_key_iv: "3a7109627b0f668c1c2022bcc22dbfb8"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.054111 11968 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:27.057174 11988 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.058723 11968 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:27.058858 11968 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "aea436054f314a08b209726d18174f89"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "cc93097d5e833df609a2539a2318d96b"
server_key_iv: "3a7109627b0f668c1c2022bcc22dbfb8"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.058995 11968 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:27.077505 11968 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:27.081276 11968 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:27.081491 11968 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:27.082296 11968 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:27.083372 11968 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:27.083427 11968 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.083509 11968 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:27.083541 11968 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.093071 11968 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:35307
I20260807 00:02:27.093116 12157 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:35307 every 8 connection(s)
I20260807 00:02:27.094085 11968 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:27.095872 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 11968
I20260807 00:02:27.096000 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:27.096347 31537 external_mini_cluster.cc:1428] Setting key e6b9235774a917dc238879b00932f341
Aug 07 00:02:27 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:27.108530 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:27.096131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:44949 (local address 127.30.204.126:43273)
0807 00:02:27.096336 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:27.096341 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:27.097577 (+ 1236us) server_negotiation.cc:407] Connection header received
0807 00:02:27.098541 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:27.098544 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:27.098608 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:27.098682 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:27.100017 (+ 1335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.100559 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.101749 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.101988 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.102084 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:27.104681 (+ 2597us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:27.104684 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:27.104686 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:27.104714 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:27.106122 (+ 1408us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:27.106684 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:27.106687 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:27.106689 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:27.106784 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:27.107132 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:27.107135 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:27.107137 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:27.107340 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:27.107445 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:27.108097 (+ 652us) server_negotiation.cc:299] Negotiation successful
0807 00:02:27.108336 (+ 239us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:02:27.109577 12162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:27.096601 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:44949)
0807 00:02:27.097410 (+ 809us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:27.097472 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:27.098328 (+ 856us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:27.098891 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:27.098899 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:27.099286 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:27.099866 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:27.099881 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.100699 (+ 818us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.100702 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:27.101619 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:27.101630 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.102158 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.102163 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:27.102331 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:27.102928 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:27.102947 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:27.104478 (+ 1531us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:27.106250 (+ 1772us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:27.106257 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:27.106276 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:27.106582 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:27.106873 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:27.106876 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:27.106878 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:27.107039 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:27.107452 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:27.107486 (+ 34us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:27.107774 (+ 288us) client_negotiation.cc:769] Sending connection context
0807 00:02:27.107978 (+ 204us) client_negotiation.cc:240] Negotiation successful
0807 00:02:27.108400 (+ 422us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":139,"threads_started":1}
I20260807 00:02:27.110947 12159 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43273
I20260807 00:02:27.111272 12159 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:27.111964 12159 heartbeater.cc:511] Master 127.30.204.126:43273 requested a full tablet report, sending...
I20260807 00:02:27.113678 11874 ts_manager.cc:194] Registered new tserver with Master: aea436054f314a08b209726d18174f89 (127.30.204.65:35307)
I20260807 00:02:27.114887 11874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:44949
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:02:27.127372 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43273
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:27.247655 12165 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:27.247954 12165 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:27.248001 12165 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:27.252190 12165 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:27.252281 12165 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:27.252343 12165 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:27.258020 12165 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12165
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:27.259207 12165 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:27.260411 12165 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:27.270977 12180 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:27.271054 12165 server_base.cc:1034] running on GCE node
W20260807 00:02:27.270967 12184 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:27.270967 12181 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:27.271792 12165 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:27.272517 12165 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:27.273689 12165 hybrid_clock.cc:648] HybridClock initialized: now 1786060947273659 us; error 39 us; skew 500 ppm
Aug 07 00:02:27 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:27.276552 12165 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:02:27.277520 12165 webserver.cc:460] Webserver started at http://127.30.204.66:32857/ using document root <none> and password file <none>
I20260807 00:02:27.278079 12165 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:27.278143 12165 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:27.278337 12165 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:27.280682 12165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "1ebc73d910f742b3ba66b6d148921ec5"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a6958c17cc63875ff5dd5082382f9f31"
server_key_iv: "db080b4a92c4ccff5d621978502b6708"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.281255 12165 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "1ebc73d910f742b3ba66b6d148921ec5"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a6958c17cc63875ff5dd5082382f9f31"
server_key_iv: "db080b4a92c4ccff5d621978502b6708"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.285300 12165 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:02:27.288280 12204 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.289759 12165 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:27.289893 12165 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "1ebc73d910f742b3ba66b6d148921ec5"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a6958c17cc63875ff5dd5082382f9f31"
server_key_iv: "db080b4a92c4ccff5d621978502b6708"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.290027 12165 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:27.334618 12165 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:27.338383 12165 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:27.338587 12165 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:27.339282 12165 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:27.340279 12165 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:27.340328 12165 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.340372 12165 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:27.340399 12165 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.349459 12165 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:43191
I20260807 00:02:27.349529 12391 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:43191 every 8 connection(s)
I20260807 00:02:27.350490 12165 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:27.356871 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 12165
I20260807 00:02:27.356994 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:27.357251 31537 external_mini_cluster.cc:1428] Setting key 8cbfa63de649ad75dff77aa81205b51b
Aug 07 00:02:27 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:27.365875 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:27.352631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:46107 (local address 127.30.204.126:43273)
0807 00:02:27.352845 (+ 214us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:27.352849 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:27.354274 (+ 1425us) server_negotiation.cc:407] Connection header received
0807 00:02:27.355219 (+ 945us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:27.355222 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:27.355279 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:27.355364 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:27.356779 (+ 1415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.357428 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.358863 (+ 1435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.359114 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.359208 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:27.362166 (+ 2958us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:27.362170 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:27.362172 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:27.362201 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:27.363672 (+ 1471us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:27.364224 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:27.364228 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:27.364230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:27.364316 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:27.364670 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:27.364674 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:27.364676 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:27.364881 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:27.364975 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:27.365543 (+ 568us) server_negotiation.cc:299] Negotiation successful
0807 00:02:27.365700 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93}
I20260807 00:02:27.366788 12397 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:27.353186 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:46107)
0807 00:02:27.354079 (+ 893us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:27.354139 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:27.354984 (+ 845us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:27.355580 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:27.355588 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:27.355986 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:27.356565 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:27.356585 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.357583 (+ 998us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.357587 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:27.358714 (+ 1127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:27.358726 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.359295 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.359300 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:27.359540 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:27.360211 (+ 671us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:27.360234 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:27.361956 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:27.363798 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:27.363806 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:27.363824 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:27.364120 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:27.364407 (+ 287us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:27.364410 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:27.364412 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:27.364578 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:27.364991 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:27.364999 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:27.365294 (+ 295us) client_negotiation.cc:769] Sending connection context
0807 00:02:27.365481 (+ 187us) client_negotiation.cc:240] Negotiation successful
0807 00:02:27.365860 (+ 379us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":664,"thread_start_us":153,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:02:27.368124 12392 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43273
I20260807 00:02:27.368397 12392 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:27.369005 12392 heartbeater.cc:511] Master 127.30.204.126:43273 requested a full tablet report, sending...
I20260807 00:02:27.370378 11874 ts_manager.cc:194] Registered new tserver with Master: 1ebc73d910f742b3ba66b6d148921ec5 (127.30.204.66:43191)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:02:27.371210 11874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:46107
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:02:27.388403 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43273
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:27.506601 12401 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:27.506891 12401 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:27.506937 12401 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:27.511111 12401 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:27.511191 12401 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:27.511256 12401 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:27.516884 12401 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12401
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:27.518026 12401 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:27.519148 12401 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:27.529726 12408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:27.529726 12411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:27.529748 12409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:27.530223 12401 server_base.cc:1034] running on GCE node
I20260807 00:02:27.530720 12401 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:27.531422 12401 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:27.532656 12401 hybrid_clock.cc:648] HybridClock initialized: now 1786060947532642 us; error 34 us; skew 500 ppm
Aug 07 00:02:27 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:27.535751 12401 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:02:27.536890 12401 webserver.cc:460] Webserver started at http://127.30.204.67:45817/ using document root <none> and password file <none>
I20260807 00:02:27.537536 12401 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:27.537613 12401 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:27.537849 12401 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:27.540127 12401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "0d2e5062cf134c4eb273fe4b7d180eaa"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a929708e8921ffd0f7e65501c5c3da6e"
server_key_iv: "2d698e2884033a8e30674849d187a6c1"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.540717 12401 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "0d2e5062cf134c4eb273fe4b7d180eaa"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a929708e8921ffd0f7e65501c5c3da6e"
server_key_iv: "2d698e2884033a8e30674849d187a6c1"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.544862 12401 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 00:02:27.547858 12421 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.549358 12401 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:27.549530 12401 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "0d2e5062cf134c4eb273fe4b7d180eaa"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a929708e8921ffd0f7e65501c5c3da6e"
server_key_iv: "2d698e2884033a8e30674849d187a6c1"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.549665 12401 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:27.568348 12401 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:27.572088 12401 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:27.572304 12401 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:27.572988 12401 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:27.573935 12401 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:27.573989 12401 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.574049 12401 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:27.574079 12401 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.583200 12401 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:46037
I20260807 00:02:27.583223 12600 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:46037 every 8 connection(s)
I20260807 00:02:27.584242 12401 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:27.586655 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 12401
I20260807 00:02:27.586803 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:27.587145 31537 external_mini_cluster.cc:1428] Setting key 83035aa4a30bd5faddcc7f2befe9f044
Aug 07 00:02:27 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:27.599143 11936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:27.586240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:33623 (local address 127.30.204.126:43273)
0807 00:02:27.586396 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:27.586408 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:27.587391 (+ 983us) server_negotiation.cc:407] Connection header received
0807 00:02:27.589009 (+ 1618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:27.589012 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:27.589072 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:27.589149 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:27.590489 (+ 1340us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.591025 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.592346 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.592570 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.592662 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:27.595657 (+ 2995us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:27.595660 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:27.595662 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:27.595695 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:27.596924 (+ 1229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:27.597492 (+ 568us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:27.597495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:27.597496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:27.597583 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:27.597957 (+ 374us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:27.597960 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:27.597961 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:27.598168 (+ 207us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:27.598257 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:27.598820 (+ 563us) server_negotiation.cc:299] Negotiation successful
0807 00:02:27.598964 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:02:27.600167 12603 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:27.586618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:33623)
0807 00:02:27.587188 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:27.587260 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:27.588787 (+ 1527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:27.589360 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:27.589368 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:27.589793 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:27.590335 (+ 542us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:27.590351 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.591149 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.591152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:27.592161 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:27.592173 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:27.592732 (+ 559us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:27.592736 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:27.592913 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:27.593556 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:27.593575 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:27.595378 (+ 1803us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:27.597055 (+ 1677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:27.597063 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:27.597083 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:27.597375 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:27.597680 (+ 305us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:27.597683 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:27.597686 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:27.597852 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:27.598275 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:27.598283 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:27.598561 (+ 278us) client_negotiation.cc:769] Sending connection context
0807 00:02:27.598742 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 00:02:27.599119 (+ 377us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":140,"threads_started":1}
I20260807 00:02:27.601285 12601 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43273
I20260807 00:02:27.601542 12601 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:27.602041 12601 heartbeater.cc:511] Master 127.30.204.126:43273 requested a full tablet report, sending...
I20260807 00:02:27.603092 11874 ts_manager.cc:194] Registered new tserver with Master: 0d2e5062cf134c4eb273fe4b7d180eaa (127.30.204.67:46037)
I20260807 00:02:27.603709 11874 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:33623
I20260807 00:02:27.613727 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab.
Entry for principal oryx/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab.
WARNING: no policy specified for oryx/127.30.204.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab.
Entry for principal oryx/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab.
WARNING: no policy specified for oryx/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab.
Entry for principal oryx/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab.
I20260807 00:02:27.698624 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 11968
I20260807 00:02:27.709506 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 12165
I20260807 00:02:27.730327 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 12401
I20260807 00:02:27.748809 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 11825
I20260807 00:02:27.767603 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43273
--webserver_interface=127.30.204.126
--webserver_port=38101
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:43273
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:27.887285 12615 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:27.887663 12615 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:27.887710 12615 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:27.891963 12615 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:27.892051 12615 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:27.892069 12615 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:27.892086 12615 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:27.892100 12615 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:27.897982 12615 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:43273
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43273
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=38101
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12615
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:27.899230 12615 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:27.900394 12615 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:27.909785 12626 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:27.909739 12630 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:27.909739 12625 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:27.909770 12615 server_base.cc:1034] running on GCE node
I20260807 00:02:27.910669 12615 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:27.911983 12615 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:27.913187 12615 hybrid_clock.cc:648] HybridClock initialized: now 1786060947913165 us; error 39 us; skew 500 ppm
Aug 07 00:02:27 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060947, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:27.916167 12615 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:27.917188 12615 webserver.cc:460] Webserver started at http://127.30.204.126:38101/ using document root <none> and password file <none>
I20260807 00:02:27.917764 12615 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:27.917845 12615 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:27.922179 12615 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:02:27.924762 12642 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:27.926293 12615 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:27.926461 12615 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "198b6c0493024d229b5591347cf6fb4a"
format_stamp: "Formatted at 2026-08-07 00:02:26 on dist-test-slave-pww0"
server_key: "2de69045333ce7150ca56e207fd935ea"
server_key_iv: "65d8030ec4cb6d709702a35061051173"
server_key_version: "encryptionkey@0"
I20260807 00:02:27.927088 12615 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:27.945014 12615 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:27.948509 12615 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:27.948715 12615 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:27.956849 12615 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:43273
I20260807 00:02:27.957002 12714 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:43273 every 8 connection(s)
I20260807 00:02:27.958072 12615 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:27.962880 12715 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:02:27.964882 12715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Bootstrap starting.
I20260807 00:02:27.965955 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 12615
I20260807 00:02:27.966486 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:35307
--local_ip_for_outbound_sockets=127.30.204.65
--tserver_master_addrs=127.30.204.126:43273
--webserver_port=39131
--webserver_interface=127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:02:27.984244 12715 log.cc:824] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:27.991814 12715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:02:27.992214 12715 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Bootstrap complete.
I20260807 00:02:27.996944 12715 raft_consensus.cc:348] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:27.997453 12715 raft_consensus.cc:729] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 198b6c0493024d229b5591347cf6fb4a, State: Initialized, Role: FOLLOWER
I20260807 00:02:27.998096 12715 consensus_queue.cc:260] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:27.998278 12715 raft_consensus.cc:388] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:27.998358 12715 raft_consensus.cc:482] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:27.998452 12715 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:02:28.001305 12715 raft_consensus.cc:504] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:28.001693 12715 leader_election.cc:302] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 198b6c0493024d229b5591347cf6fb4a; no voters:
I20260807 00:02:28.002061 12715 leader_election.cc:288] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:02:28.002187 12725 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:02:28.002375 12725 raft_consensus.cc:686] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [term 2 LEADER]: Becoming Leader. State: Replica: 198b6c0493024d229b5591347cf6fb4a, State: Running, Role: LEADER
I20260807 00:02:28.002718 12725 consensus_queue.cc:237] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } }
I20260807 00:02:28.003527 12715 sys_catalog.cc:564] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:28.003932 12729 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 198b6c0493024d229b5591347cf6fb4a. Latest consensus state: current_term: 2 leader_uuid: "198b6c0493024d229b5591347cf6fb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } } }
I20260807 00:02:28.004070 12729 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:28.004280 12726 sys_catalog.cc:455] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "198b6c0493024d229b5591347cf6fb4a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "198b6c0493024d229b5591347cf6fb4a" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43273 } } }
I20260807 00:02:28.004387 12726 sys_catalog.cc:458] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:28.004516 12738 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:28.008281 12738 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:28.009013 12738 catalog_manager.cc:1259] Loaded cluster ID: 3c83c89e0ac2479eacc8df904cd083b7
I20260807 00:02:28.009078 12738 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:28.010864 12738 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:28.011516 12738 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 198b6c0493024d229b5591347cf6fb4a: Loaded TSK: 0
I20260807 00:02:28.012697 12738 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:02:28.124200 12722 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:28.124598 12722 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:28.124665 12722 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:28.130645 12722 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:28.130760 12722 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:28.130857 12722 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:28.138979 12722 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.65.keytab
--principal=oryx/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:35307
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=39131
--tserver_master_addrs=127.30.204.126:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12722
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:28.140569 12722 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:28.142033 12722 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:28.156200 12766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:28.156348 12762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:28.156719 12769 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:28.156670 12722 server_base.cc:1034] running on GCE node
I20260807 00:02:28.157202 12722 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:28.157994 12722 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:28.159214 12722 hybrid_clock.cc:648] HybridClock initialized: now 1786060948159180 us; error 55 us; skew 500 ppm
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060948, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:28.162779 12722 init.cc:377] Logged in from keytab as oryx/127.30.204.65@KRBTEST.COM (short username oryx)
I20260807 00:02:28.163987 12722 webserver.cc:460] Webserver started at http://127.30.204.65:39131/ using document root <none> and password file <none>
I20260807 00:02:28.164597 12722 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:28.164692 12722 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:28.169186 12722 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:02:28.171782 12784 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.173370 12722 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:28.173533 12722 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "aea436054f314a08b209726d18174f89"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "cc93097d5e833df609a2539a2318d96b"
server_key_iv: "3a7109627b0f668c1c2022bcc22dbfb8"
server_key_version: "encryptionkey@0"
I20260807 00:02:28.174216 12722 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:28.192284 12722 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:28.196547 12722 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:28.196756 12722 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:28.197458 12722 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:28.198426 12722 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:28.198482 12722 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.198540 12722 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:28.198568 12722 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.208175 12722 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:35307
I20260807 00:02:28.208217 12942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:35307 every 8 connection(s)
I20260807 00:02:28.209332 12722 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:28.216151 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 12722
I20260807 00:02:28.216601 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:43191
--local_ip_for_outbound_sockets=127.30.204.66
--tserver_master_addrs=127.30.204.126:43273
--webserver_port=32857
--webserver_interface=127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:28.224896 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.211959 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:47163)
0807 00:02:28.212733 (+ 774us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.212778 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.213642 (+ 864us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.214852 (+ 1210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.214861 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.215337 (+ 476us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.215868 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.215890 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.217535 (+ 1645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.217539 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.218774 (+ 1235us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.218788 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.220058 (+ 1270us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.220062 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.220241 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.221173 (+ 932us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.221198 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.223998 (+ 2800us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":580,"thread_start_us":171,"threads_started":1}
W20260807 00:02:28.225479 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.211825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:47163 (local address 127.30.204.126:43273)
0807 00:02:28.212426 (+ 601us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.212446 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.212912 (+ 466us) server_negotiation.cc:407] Connection header received
0807 00:02:28.214075 (+ 1163us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.214086 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.214420 (+ 334us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.214752 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.216149 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.217361 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.218948 (+ 1587us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.219846 (+ 898us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.220041 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.224397 (+ 4356us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:47163: BlockingRecv error: recv got EOF from 127.30.204.65:47163 (error 108)
Metrics: {"server-negotiator.queue_time_us":393,"thread_start_us":142,"threads_started":1}
W20260807 00:02:28.225590 12943 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:43273 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.204.126:43273: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:28.231594 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.226078 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:46405)
0807 00:02:28.226201 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.226217 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.226315 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.226555 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.226558 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.226730 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.226837 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.226843 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.227630 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.227633 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.228202 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.228211 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.228659 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.228664 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.228753 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.229280 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.229292 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.231409 (+ 2117us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 00:02:28.231766 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.226191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:46405 (local address 127.30.204.126:43273)
0807 00:02:28.226311 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.226315 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.226331 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:28.226403 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.226406 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.226454 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.226523 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.226959 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.227524 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.228323 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.228548 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.228639 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.231650 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:46405: BlockingRecv error: recv got EOF from 127.30.204.65:46405 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:28.237437 12948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.232293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:33587)
0807 00:02:28.232413 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.232429 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.232521 (+ 92us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.232776 (+ 255us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.232779 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.232936 (+ 157us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.233022 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.233028 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.233756 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.233760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.234303 (+ 543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.234312 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.234743 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.234747 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.234823 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.235375 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.235388 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.237290 (+ 1902us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 00:02:28.237558 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.232363 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:33587 (local address 127.30.204.126:43273)
0807 00:02:28.232469 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.232473 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.232487 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:28.232605 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.232608 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.232675 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.232743 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.233135 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.233651 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.234417 (+ 766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.234640 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.234729 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.237454 (+ 2725us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:33587: BlockingRecv error: recv got EOF from 127.30.204.65:33587 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
W20260807 00:02:28.237805 12943 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:02:28.339169 12949 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:28.339520 12949 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:28.339591 12949 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:28.343842 12949 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:28.343923 12949 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:28.343986 12949 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:28.349658 12949 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.66.keytab
--principal=oryx/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:43191
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=32857
--tserver_master_addrs=127.30.204.126:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.12949
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:28.351048 12949 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:28.352214 12949 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:28.362741 12963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:28.362742 12965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:28.362742 12962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:28.363152 12949 server_base.cc:1034] running on GCE node
I20260807 00:02:28.363692 12949 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:28.364365 12949 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:28.365550 12949 hybrid_clock.cc:648] HybridClock initialized: now 1786060948365527 us; error 63 us; skew 500 ppm
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060948, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:28.368571 12949 init.cc:377] Logged in from keytab as oryx/127.30.204.66@KRBTEST.COM (short username oryx)
I20260807 00:02:28.369534 12949 webserver.cc:460] Webserver started at http://127.30.204.66:32857/ using document root <none> and password file <none>
I20260807 00:02:28.370110 12949 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:28.370189 12949 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:28.374557 12949 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:28.377079 12983 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.378597 12949 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:28.378721 12949 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "1ebc73d910f742b3ba66b6d148921ec5"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a6958c17cc63875ff5dd5082382f9f31"
server_key_iv: "db080b4a92c4ccff5d621978502b6708"
server_key_version: "encryptionkey@0"
I20260807 00:02:28.379369 12949 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:28.390903 12949 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:28.394351 12949 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:28.394541 12949 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:28.395179 12949 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:28.396102 12949 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:28.396160 12949 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.396205 12949 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:28.396234 12949 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.405453 12949 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:43191
I20260807 00:02:28.405489 13151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:43191 every 8 connection(s)
I20260807 00:02:28.406569 12949 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:28.415697 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 12949
I20260807 00:02:28.416218 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:46037
--local_ip_for_outbound_sockets=127.30.204.67
--tserver_master_addrs=127.30.204.126:43273
--webserver_port=45817
--webserver_interface=127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46785
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:02:28.418617 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.408590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:40175 (local address 127.30.204.126:43273)
0807 00:02:28.408771 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.408776 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.409934 (+ 1158us) server_negotiation.cc:407] Connection header received
0807 00:02:28.410834 (+ 900us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.410837 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.410892 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.410968 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.412299 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.412862 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.414046 (+ 1184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.414298 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.414391 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.418482 (+ 4091us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:40175: BlockingRecv error: recv got EOF from 127.30.204.66:40175 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:02:28.419310 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.409015 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:40175)
0807 00:02:28.409774 (+ 759us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.409823 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.410651 (+ 828us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.411184 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.411192 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.411607 (+ 415us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.412127 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.412144 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.412986 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.412990 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.413909 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.413920 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.414438 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.414440 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.414587 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.415166 (+ 579us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.415183 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.418314 (+ 3131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":140,"threads_started":1}
W20260807 00:02:28.419834 13154 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:43273 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.204.126:43273: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:28.425611 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.420314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:48797)
0807 00:02:28.420466 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.420483 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.420574 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.420861 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.420864 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.421006 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.421106 (+ 100us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.421112 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.422149 (+ 1037us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.422152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.422711 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.422720 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.423261 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.423264 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.423330 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.423839 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.423850 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.425502 (+ 1652us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":67}
W20260807 00:02:28.425836 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.420438 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:48797 (local address 127.30.204.126:43273)
0807 00:02:28.420582 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.420587 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.420629 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:02:28.420681 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.420684 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.420732 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.420810 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.421240 (+ 430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.422033 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.422823 (+ 790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.423159 (+ 336us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.423283 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.425703 (+ 2420us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:48797: BlockingRecv error: recv got EOF from 127.30.204.66:48797 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:28.432274 13157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.426570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:32817)
0807 00:02:28.426709 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.426724 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.426797 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.427067 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.427069 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.427222 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.427303 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.427308 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.428417 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.428420 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.428993 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.429003 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.429631 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.429634 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.429713 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.430236 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.430248 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.432156 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 00:02:28.432461 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.426681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:32817 (local address 127.30.204.126:43273)
0807 00:02:28.426810 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.426814 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.426830 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:28.426876 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.426879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.426928 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.427018 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.427417 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.428231 (+ 814us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.429123 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.429488 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.429597 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.432323 (+ 2726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:32817: BlockingRecv error: recv got EOF from 127.30.204.66:32817 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
W20260807 00:02:28.432749 13154 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:02:28.537307 13161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:28.537606 13161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:28.537652 13161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:28.541865 13161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:28.541949 13161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:28.542021 13161 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:28.547837 13161 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46785
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/krb5kdc/oryx_127.30.204.67.keytab
--principal=oryx/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:46037
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=45817
--tserver_master_addrs=127.30.204.126:43273
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:28.549013 13161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:28.550127 13161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:28.561215 13168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:28.561210 13171 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:28.561494 13161 server_base.cc:1034] running on GCE node
W20260807 00:02:28.561343 13174 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:28.561997 13161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:28.562682 13161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:28.563854 13161 hybrid_clock.cc:648] HybridClock initialized: now 1786060948563837 us; error 29 us; skew 500 ppm
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060948, etypes {rep=17 tkt=17 ses=17}, oryx/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:28.566850 13161 init.cc:377] Logged in from keytab as oryx/127.30.204.67@KRBTEST.COM (short username oryx)
I20260807 00:02:28.567826 13161 webserver.cc:460] Webserver started at http://127.30.204.67:45817/ using document root <none> and password file <none>
I20260807 00:02:28.568374 13161 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:28.568449 13161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:28.572708 13161 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.003s
I20260807 00:02:28.575114 13188 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.576588 13161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:28.576745 13161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "0d2e5062cf134c4eb273fe4b7d180eaa"
format_stamp: "Formatted at 2026-08-07 00:02:27 on dist-test-slave-pww0"
server_key: "a929708e8921ffd0f7e65501c5c3da6e"
server_key_iv: "2d698e2884033a8e30674849d187a6c1"
server_key_version: "encryptionkey@0"
I20260807 00:02:28.577344 13161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:28.589862 13161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:28.593282 13161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:28.593480 13161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:28.594156 13161 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:28.595075 13161 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:28.595125 13161 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.595168 13161 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:28.595196 13161 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:28.604281 13161 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:46037
I20260807 00:02:28.604369 13362 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:46037 every 8 connection(s)
I20260807 00:02:28.605491 13161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:28.615109 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 13161
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:02:28.617136 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.607541 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:47573 (local address 127.30.204.126:43273)
0807 00:02:28.607695 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.607699 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.608824 (+ 1125us) server_negotiation.cc:407] Connection header received
0807 00:02:28.609709 (+ 885us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.609712 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.609764 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.609842 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.611077 (+ 1235us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.611677 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.612924 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.613165 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.613260 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.617013 (+ 3753us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:47573: BlockingRecv error: recv got EOF from 127.30.204.67:47573 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:02:28.617887 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.607958 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:47573)
0807 00:02:28.608661 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.608705 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.609494 (+ 789us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.610006 (+ 512us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.610012 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.610417 (+ 405us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.610924 (+ 507us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.610939 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.611824 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.611827 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.612775 (+ 948us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.612786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.613356 (+ 570us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.613359 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.613522 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.614192 (+ 670us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.614212 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.616866 (+ 2654us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":491,"thread_start_us":120,"threads_started":1}
W20260807 00:02:28.618412 13364 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:43273 (0 consecutive failures): Not authorized: Failed to ping master at 127.30.204.126:43273: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:28.624337 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.618885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:38293)
0807 00:02:28.619001 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.619016 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.619086 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.619359 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.619362 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.619537 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.619649 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.619655 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.620543 (+ 888us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.620547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.621202 (+ 655us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.621213 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.621768 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.621772 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.621891 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.622311 (+ 420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.622327 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.624223 (+ 1896us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 00:02:28.624476 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.618963 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:38293 (local address 127.30.204.126:43273)
0807 00:02:28.619091 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.619096 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.619134 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:02:28.619170 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.619173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.619218 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.619289 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.619764 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.620415 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.621340 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.621654 (+ 314us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.621752 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.624365 (+ 2613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:38293: BlockingRecv error: recv got EOF from 127.30.204.67:38293 (error 108)
Metrics: {"mutex_wait_us":34,"server-negotiator.queue_time_us":36}
I20260807 00:02:28.626567 13371 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.616306 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38992 (local address 127.30.204.126:43273)
0807 00:02:28.616582 (+ 276us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.616585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.616866 (+ 281us) server_negotiation.cc:407] Connection header received
0807 00:02:28.617011 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.617014 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.617070 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.617167 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.617994 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.618632 (+ 638us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.619762 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.620088 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.620173 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.621805 (+ 1632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:28.621813 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:28.621839 (+ 26us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:28.621878 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:28.623810 (+ 1932us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:28.624660 (+ 850us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:28.624667 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:28.624675 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:28.624832 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:28.625123 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:28.625127 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:28.625129 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:28.625697 (+ 568us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:28.625872 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:28.626197 (+ 325us) server_negotiation.cc:299] Negotiation successful
0807 00:02:28.626411 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":32,"server-negotiator.queue_time_us":182,"thread_start_us":85,"threads_started":1}
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:28 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:28.630336 13368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:28.624933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:53747)
0807 00:02:28.625033 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:28.625048 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:28.625124 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:28.625346 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:28.625349 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:28.625501 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:28.625605 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.625612 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.626546 (+ 934us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.626550 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.627146 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:28.627157 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.627744 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.627749 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:28.627839 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.628321 (+ 482us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:28.628336 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:28.630208 (+ 1872us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 00:02:28.630517 12945 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:28.625005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:53747 (local address 127.30.204.126:43273)
0807 00:02:28.625171 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:28.625175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:28.625187 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:02:28.625215 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:28.625217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:28.625251 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:28.625307 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:28.625720 (+ 413us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.626423 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.627256 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:28.627619 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:28.627722 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:28.630397 (+ 2675us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:53747: BlockingRecv error: recv got EOF from 127.30.204.67:53747 (error 108)
Metrics: {"server-negotiator.queue_time_us":87}
W20260807 00:02:28.631325 13364 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 00:02:29 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:29 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:29.244412 13372 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:29.238537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:38969)
0807 00:02:29.238845 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:29.238863 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:29.238980 (+ 117us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:29.239285 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:29.239288 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:29.239549 (+ 261us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:29.239720 (+ 171us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:29.239727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.240471 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.240474 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:29.241094 (+ 620us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:29.241104 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.241569 (+ 465us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.241572 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:29.241680 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:29.242184 (+ 504us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:29.242196 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:29.244277 (+ 2081us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":91,"threads_started":1}
W20260807 00:02:29.244611 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:29.238605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:38969 (local address 127.30.204.126:43273)
0807 00:02:29.238923 (+ 318us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:29.238926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:29.238940 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:29.239083 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:29.239086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:29.239142 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:29.239251 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:29.239821 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.240358 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.241212 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.241460 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.241536 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:29.244454 (+ 2918us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:38969: BlockingRecv error: recv got EOF from 127.30.204.65:38969 (error 108)
Metrics: {"server-negotiator.queue_time_us":232,"thread_start_us":107,"threads_started":1}
Aug 07 00:02:29 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:29 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:29.439298 13375 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:29.433489 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:40341)
0807 00:02:29.433774 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:29.433788 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:29.433923 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:29.434248 (+ 325us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:29.434250 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:29.434452 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:29.434583 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:29.434590 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.435553 (+ 963us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.435556 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:29.436162 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:29.436172 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.436652 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.436655 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:29.436724 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:29.437267 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:29.437280 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:29.439160 (+ 1880us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":90,"threads_started":1}
W20260807 00:02:29.439507 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:29.433579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:40341 (local address 127.30.204.126:43273)
0807 00:02:29.433740 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:29.433744 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:29.433901 (+ 157us) server_negotiation.cc:407] Connection header received
0807 00:02:29.434043 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:29.434046 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:29.434109 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:29.434205 (+ 96us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:29.434728 (+ 523us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.435415 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.436294 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.436518 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.436605 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:29.439356 (+ 2751us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:40341: BlockingRecv error: recv got EOF from 127.30.204.66:40341 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
Aug 07 00:02:29 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:29 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:29.637439 13376 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:29.631968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:57561)
0807 00:02:29.632290 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:29.632304 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:29.632423 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:29.632681 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:29.632684 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:29.632894 (+ 210us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:29.633014 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:29.633020 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.633784 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.633788 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:29.634368 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:29.634377 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.634826 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.634830 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:29.634933 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:29.635411 (+ 478us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:29.635423 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:29.637280 (+ 1857us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":100,"threads_started":1}
W20260807 00:02:29.637579 13374 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:29.632028 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:57561 (local address 127.30.204.126:43273)
0807 00:02:29.632189 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:29.632195 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:29.632381 (+ 186us) server_negotiation.cc:407] Connection header received
0807 00:02:29.632519 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:29.632523 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:29.632573 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:29.632643 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:29.633138 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.633671 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.634489 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:29.634716 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:29.634805 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:29.637468 (+ 2663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:57561: BlockingRecv error: recv got EOF from 127.30.204.67:57561 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Aug 07 00:02:30 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:30 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:30.251624 13379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:30.245421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:54179)
0807 00:02:30.245749 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:30.245766 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:30.245879 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:30.246171 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:30.246174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:30.246375 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:30.246497 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:30.246504 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.247319 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.247322 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:30.247991 (+ 669us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:30.248003 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.248486 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.248490 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:30.248577 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:30.249435 (+ 858us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:30.249447 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:30.251416 (+ 1969us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":77,"threads_started":1}
W20260807 00:02:30.251730 13380 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:30.245581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:54179 (local address 127.30.204.126:43273)
0807 00:02:30.245874 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:30.245878 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:30.245894 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:30.245956 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:30.245959 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:30.246003 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:30.246081 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:30.246637 (+ 556us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.247192 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.248105 (+ 913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.248347 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.248432 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:30.251597 (+ 3165us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:54179: BlockingRecv error: recv got EOF from 127.30.204.65:54179 (error 108)
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":106,"threads_started":1}
Aug 07 00:02:30 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:30 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:30.446130 13381 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:30.440323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:60993)
0807 00:02:30.440633 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:30.440656 (+ 23us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:30.440767 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:30.441016 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:30.441018 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:30.441205 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:30.441310 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:30.441316 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.442140 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.442144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:30.442737 (+ 593us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:30.442747 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.443202 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.443205 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:30.443293 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:30.443957 (+ 664us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:30.443971 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:30.445978 (+ 2007us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":103,"threads_started":1}
W20260807 00:02:30.446230 13380 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:30.440463 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:60993 (local address 127.30.204.126:43273)
0807 00:02:30.440639 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:30.440643 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:30.440732 (+ 89us) server_negotiation.cc:407] Connection header received
0807 00:02:30.440848 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:30.440850 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:30.440902 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:30.440990 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:30.441452 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.442020 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.442850 (+ 830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.443086 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.443163 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:30.446125 (+ 2962us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:60993: BlockingRecv error: recv got EOF from 127.30.204.66:60993 (error 108)
Metrics: {"server-negotiator.queue_time_us":69}
Aug 07 00:02:30 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:30 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:30.644390 13382 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:30.638381 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:42921)
0807 00:02:30.638685 (+ 304us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:30.638704 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:30.638818 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:30.639081 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:30.639084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:30.639299 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:30.639406 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:30.639412 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.640234 (+ 822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.640239 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:30.640821 (+ 582us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:30.640831 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.641302 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.641306 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:30.641391 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:30.641962 (+ 571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:30.641976 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:30.644184 (+ 2208us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":85,"threads_started":1}
W20260807 00:02:30.644533 13380 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:30.638593 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:42921 (local address 127.30.204.126:43273)
0807 00:02:30.638739 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:30.638744 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:30.638758 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:30.638916 (+ 158us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:30.638919 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:30.638970 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:30.639056 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:30.639570 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.640115 (+ 545us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.640949 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:30.641185 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:30.641281 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:30.644418 (+ 3137us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:42921: BlockingRecv error: recv got EOF from 127.30.204.67:42921 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 00:02:31 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:31 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:31.258652 13383 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:31.252533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:38307)
0807 00:02:31.252882 (+ 349us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:31.252897 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:31.253005 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:31.253275 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:31.253277 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:31.253467 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:31.253576 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:31.253583 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.254350 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.254354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:31.255253 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:31.255266 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.255798 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.255802 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:31.255879 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:31.256445 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:31.256459 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:31.258454 (+ 1995us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":263,"thread_start_us":132,"threads_started":1}
W20260807 00:02:31.258801 13384 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:31.252677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:38307 (local address 127.30.204.126:43273)
0807 00:02:31.252938 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:31.252941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:31.252958 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:02:31.253108 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:31.253111 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:31.253164 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:31.253252 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:31.253674 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.254221 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.255386 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.255653 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.255744 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:31.258666 (+ 2922us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:38307: BlockingRecv error: recv got EOF from 127.30.204.65:38307 (error 108)
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":82,"threads_started":1}
Aug 07 00:02:31 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:31 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:31.452741 13386 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:31.446995 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:50861)
0807 00:02:31.447325 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:31.447342 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:31.447520 (+ 178us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:31.447812 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:31.447815 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:31.448032 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:31.448138 (+ 106us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:31.448144 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.448931 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.448934 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:31.449501 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:31.449510 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.449967 (+ 457us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.449971 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:31.450056 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:31.450584 (+ 528us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:31.450598 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:31.452596 (+ 1998us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":101,"threads_started":1}
W20260807 00:02:31.453099 13384 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:31.447138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:50861 (local address 127.30.204.126:43273)
0807 00:02:31.447327 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:31.447332 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:31.447403 (+ 71us) server_negotiation.cc:407] Connection header received
0807 00:02:31.447654 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:31.447657 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:31.447710 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:31.447775 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:31.448264 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.448818 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.449625 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.449854 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.449944 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:31.452964 (+ 3020us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:50861: BlockingRecv error: recv got EOF from 127.30.204.66:50861 (error 108)
Metrics: {"server-negotiator.queue_time_us":68}
Aug 07 00:02:31 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:31 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:31.651237 13388 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:31.645370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:32929)
0807 00:02:31.645747 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:31.645762 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:31.645876 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:31.646157 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:31.646161 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:31.646354 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:31.646462 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:31.646469 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.647194 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.647196 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:31.647861 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:31.647872 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.648355 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.648359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:31.648431 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:31.648970 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:31.648983 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:31.651068 (+ 2085us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":284,"thread_start_us":96,"threads_started":1}
W20260807 00:02:31.651489 13384 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:31.645578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:32929 (local address 127.30.204.126:43273)
0807 00:02:31.645751 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:31.645756 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:31.645832 (+ 76us) server_negotiation.cc:407] Connection header received
0807 00:02:31.645954 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:31.645956 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:31.646007 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:31.646078 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:31.646562 (+ 484us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.647108 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.647985 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:31.648220 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:31.648317 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:31.651332 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:32929: BlockingRecv error: recv got EOF from 127.30.204.67:32929 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
Aug 07 00:02:32 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:32 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:32.269897 13389 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:32.259784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:52181)
0807 00:02:32.260068 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:32.260083 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:32.260199 (+ 116us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:32.260490 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:32.260493 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:32.260705 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:32.260842 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:32.260851 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.261828 (+ 977us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.261832 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:32.262420 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:32.262432 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.262870 (+ 438us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.262872 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:32.262940 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:32.267324 (+ 4384us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:32.267343 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:32.269707 (+ 2364us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":81,"threads_started":1}
W20260807 00:02:32.270077 13394 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:32.259905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:52181 (local address 127.30.204.126:43273)
0807 00:02:32.260190 (+ 285us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:32.260194 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:32.260212 (+ 18us) server_negotiation.cc:407] Connection header received
0807 00:02:32.260278 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:32.260281 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:32.260326 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:32.260418 (+ 92us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:32.260995 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.261715 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.262529 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.262771 (+ 242us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.262853 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:32.269923 (+ 7070us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:52181: BlockingRecv error: recv got EOF from 127.30.204.65:52181 (error 108)
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":92,"threads_started":1}
Aug 07 00:02:32 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:32 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:32.460109 13395 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:32.453925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:55259)
0807 00:02:32.454264 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:32.454281 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:32.454416 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:32.454695 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:32.454698 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:32.454949 (+ 251us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:32.455081 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:32.455090 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.456124 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.456128 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:32.456772 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:32.456803 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.457263 (+ 460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.457267 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:32.457349 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:32.457891 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:32.457904 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:32.459934 (+ 2030us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":126,"threads_started":1}
W20260807 00:02:32.460273 13394 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:32.454074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:55259 (local address 127.30.204.126:43273)
0807 00:02:32.454264 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:32.454270 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:32.454363 (+ 93us) server_negotiation.cc:407] Connection header received
0807 00:02:32.454513 (+ 150us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:32.454517 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:32.454565 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:32.454654 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:32.455230 (+ 576us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.456005 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.456913 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.457150 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.457255 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:32.460137 (+ 2882us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:55259: BlockingRecv error: recv got EOF from 127.30.204.66:55259 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 00:02:32 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:32 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:32.657714 13398 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:32.652214 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:43577)
0807 00:02:32.652546 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:32.652567 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:32.652692 (+ 125us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:32.652949 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:32.652952 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:32.653169 (+ 217us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:32.653271 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:32.653280 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.654071 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.654075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:32.654648 (+ 573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:32.654657 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.655105 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.655108 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:32.655192 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:32.655753 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:32.655765 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:32.657546 (+ 1781us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":94,"threads_started":1}
W20260807 00:02:32.657816 13394 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:32.652324 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:43577 (local address 127.30.204.126:43273)
0807 00:02:32.652503 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:32.652508 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:32.652641 (+ 133us) server_negotiation.cc:407] Connection header received
0807 00:02:32.652782 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:32.652785 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:32.652838 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:32.652906 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:32.653397 (+ 491us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.653959 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.654768 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:32.654993 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:32.655090 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:32.657708 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:43577: BlockingRecv error: recv got EOF from 127.30.204.67:43577 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 00:02:33 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:33 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:33.276654 13399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:33.270970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:34247)
0807 00:02:33.271271 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:33.271290 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:33.271412 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:33.271748 (+ 336us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:33.271750 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:33.271943 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:33.272045 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:33.272052 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.272836 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.272839 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:33.273563 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:33.273575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.274061 (+ 486us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.274064 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:33.274133 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:33.274619 (+ 486us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:33.274630 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:33.276527 (+ 1897us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":83,"threads_started":1}
W20260807 00:02:33.276859 13401 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:33.271144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:34247 (local address 127.30.204.126:43273)
0807 00:02:33.271418 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:33.271421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:33.271438 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:02:33.271516 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:33.271519 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:33.271568 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:33.271652 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:33.272191 (+ 539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.272739 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.273680 (+ 941us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.273955 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.274038 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:33.276717 (+ 2679us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:34247: BlockingRecv error: recv got EOF from 127.30.204.65:34247 (error 108)
Metrics: {"server-negotiator.queue_time_us":192,"thread_start_us":83,"threads_started":1}
Aug 07 00:02:33 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:33 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:33.466737 13402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:33.461117 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:45659)
0807 00:02:33.461456 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:33.461475 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:33.461596 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:33.461887 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:33.461890 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:33.462095 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:33.462203 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:33.462210 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.462989 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.462992 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:33.463611 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:33.463623 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.464090 (+ 467us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.464094 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:33.464189 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:33.464719 (+ 530us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:33.464731 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:33.466584 (+ 1853us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":101,"threads_started":1}
W20260807 00:02:33.466897 13401 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:33.461373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:45659 (local address 127.30.204.126:43273)
0807 00:02:33.461543 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:33.461547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:33.461563 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:33.461722 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:33.461724 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:33.461771 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:33.461840 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:33.462330 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.462872 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.463739 (+ 867us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.463973 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.464068 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:33.466777 (+ 2709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:45659: BlockingRecv error: recv got EOF from 127.30.204.66:45659 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Aug 07 00:02:33 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:33 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:33.664666 13403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:33.658756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:50319)
0807 00:02:33.659136 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:33.659152 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:33.659275 (+ 123us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:33.659592 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:33.659594 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:33.659810 (+ 216us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:33.659913 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:33.659919 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.660733 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.660736 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:33.661324 (+ 588us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:33.661333 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.661893 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.661897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:33.662006 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:33.662566 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:33.662578 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:33.664534 (+ 1956us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":280,"thread_start_us":114,"threads_started":1}
W20260807 00:02:33.664886 13401 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:33.658918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:50319 (local address 127.30.204.126:43273)
0807 00:02:33.659108 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:33.659112 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:33.659273 (+ 161us) server_negotiation.cc:407] Connection header received
0807 00:02:33.659352 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:33.659354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:33.659413 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:33.659524 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:33.660049 (+ 525us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.660625 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.661474 (+ 849us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:33.661723 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:33.661854 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:33.664756 (+ 2902us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:50319: BlockingRecv error: recv got EOF from 127.30.204.67:50319 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Aug 07 00:02:34 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:34 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:34.283870 13407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:34.277604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:50163)
0807 00:02:34.277952 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:34.277967 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:34.278086 (+ 119us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:34.278376 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:34.278379 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:34.278624 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:34.278741 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:34.278749 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.279654 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.279658 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:34.280242 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:34.280253 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.280837 (+ 584us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.280840 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:34.280915 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:34.281455 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:34.281490 (+ 35us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:34.283744 (+ 2254us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":123,"threads_started":1}
W20260807 00:02:34.284089 13408 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:34.277729 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:50163 (local address 127.30.204.126:43273)
0807 00:02:34.277989 (+ 260us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:34.277992 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:34.278030 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:02:34.278166 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:34.278169 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:34.278218 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:34.278300 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:34.278902 (+ 602us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.279536 (+ 634us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.280386 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.280694 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.280792 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:34.283895 (+ 3103us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:50163: BlockingRecv error: recv got EOF from 127.30.204.65:50163 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":60,"threads_started":1}
Aug 07 00:02:34 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:34 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:34.473210 13411 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:34.467647 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:42837)
0807 00:02:34.467947 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:34.467960 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:34.468068 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:34.468354 (+ 286us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:34.468357 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:34.468546 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:34.468669 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:34.468674 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.469397 (+ 723us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.469401 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:34.469985 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:34.469995 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.470491 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.470495 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:34.470569 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:34.471084 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:34.471096 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:34.473043 (+ 1947us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":121,"threads_started":1}
W20260807 00:02:34.473369 13408 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:34.467703 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:42837 (local address 127.30.204.126:43273)
0807 00:02:34.467855 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:34.467861 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:34.468029 (+ 168us) server_negotiation.cc:407] Connection header received
0807 00:02:34.468133 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:34.468135 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:34.468184 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:34.468257 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:34.468766 (+ 509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.469298 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.470099 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.470361 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.470443 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:34.473254 (+ 2811us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:42837: BlockingRecv error: recv got EOF from 127.30.204.66:42837 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 00:02:34 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:34 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:34.671283 13412 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:34.665718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:44581)
0807 00:02:34.666027 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:34.666041 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:34.666174 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:34.666425 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:34.666427 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:34.666625 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:34.666729 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:34.666735 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.667587 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.667591 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:34.668189 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:34.668200 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.668679 (+ 479us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.668682 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:34.668751 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:34.669305 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:34.669318 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:34.671149 (+ 1831us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":83,"threads_started":1}
W20260807 00:02:34.671484 13408 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:34.665812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:44581 (local address 127.30.204.126:43273)
0807 00:02:34.665974 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:34.665979 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:34.666131 (+ 152us) server_negotiation.cc:407] Connection header received
0807 00:02:34.666254 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:34.666256 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:34.666309 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:34.666386 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:34.666878 (+ 492us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.667439 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.668327 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:34.668562 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:34.668650 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:34.671320 (+ 2670us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:44581: BlockingRecv error: recv got EOF from 127.30.204.67:44581 (error 108)
Metrics: {"server-negotiator.queue_time_us":62}
Aug 07 00:02:35 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:35 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:35.290728 13413 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:35.284760 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:58667)
0807 00:02:35.285171 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:35.285187 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:35.285321 (+ 134us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:35.285659 (+ 338us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:35.285663 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:35.285920 (+ 257us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:35.286031 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:35.286039 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.286839 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.286842 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:35.287432 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:35.287443 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.287948 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.287951 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:35.288026 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:35.288593 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:35.288606 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:35.290595 (+ 1989us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":89,"threads_started":1}
W20260807 00:02:35.290925 13414 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:35.284841 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:58667 (local address 127.30.204.126:43273)
0807 00:02:35.285153 (+ 312us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:35.285158 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:35.285266 (+ 108us) server_negotiation.cc:407] Connection header received
0807 00:02:35.285421 (+ 155us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:35.285424 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:35.285484 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:35.285578 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:35.286153 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.286727 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.287570 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.287820 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.287903 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:35.290775 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:58667: BlockingRecv error: recv got EOF from 127.30.204.65:58667 (error 108)
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":89,"threads_started":1}
Aug 07 00:02:35 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:35 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:35.479697 13415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:35.474123 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:40029)
0807 00:02:35.474513 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:35.474527 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:35.474639 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:35.474872 (+ 233us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:35.474874 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:35.475067 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:35.475181 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:35.475187 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.475967 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.475970 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:35.476538 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:35.476548 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.476932 (+ 384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.476934 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:35.476996 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:35.477483 (+ 487us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:35.477496 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:35.479529 (+ 2033us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":311,"thread_start_us":103,"threads_started":1}
W20260807 00:02:35.479808 13414 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:35.474259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:40029 (local address 127.30.204.126:43273)
0807 00:02:35.474455 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:35.474461 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:35.474598 (+ 137us) server_negotiation.cc:407] Connection header received
0807 00:02:35.474720 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:35.474723 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:35.474771 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:35.474843 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:35.475275 (+ 432us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.475837 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.476628 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.476850 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.476917 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:35.479685 (+ 2768us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:40029: BlockingRecv error: recv got EOF from 127.30.204.66:40029 (error 108)
Metrics: {"server-negotiator.queue_time_us":84}
Aug 07 00:02:35 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:35 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:35.678886 13416 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:35.672317 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:55539)
0807 00:02:35.672723 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:35.672741 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:35.672854 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:35.673135 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:35.673138 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:35.673326 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:35.673433 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:35.673439 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.674361 (+ 922us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.674364 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:35.674967 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:35.674978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.675737 (+ 759us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.675741 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:35.675836 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:35.676481 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:35.676497 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:35.678721 (+ 2224us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":103,"threads_started":1}
W20260807 00:02:35.679016 13414 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:35.672442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:55539 (local address 127.30.204.126:43273)
0807 00:02:35.672576 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:35.672580 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:35.672813 (+ 233us) server_negotiation.cc:407] Connection header received
0807 00:02:35.672934 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:35.672937 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:35.672995 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:35.673076 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:35.673590 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.674212 (+ 622us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.675144 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:35.675573 (+ 429us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:35.675697 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:35.678903 (+ 3206us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:55539: BlockingRecv error: recv got EOF from 127.30.204.67:55539 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 00:02:36 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:36 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:36.297835 13418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:36.291801 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:59949)
0807 00:02:36.292069 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:36.292086 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:36.292218 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:36.292637 (+ 419us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:36.292640 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:36.292872 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:36.293012 (+ 140us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:36.293021 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.293852 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.293856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:36.294433 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:36.294443 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.294975 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.294979 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:36.295052 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:36.295624 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:36.295636 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:36.297700 (+ 2064us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":82,"threads_started":1}
W20260807 00:02:36.297996 13420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:36.291934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:59949 (local address 127.30.204.126:43273)
0807 00:02:36.292209 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:36.292213 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:36.292228 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:36.292402 (+ 174us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:36.292406 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:36.292471 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:36.292566 (+ 95us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:36.293158 (+ 592us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.293734 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.294575 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.294824 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.294920 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:36.297864 (+ 2944us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:59949: BlockingRecv error: recv got EOF from 127.30.204.65:59949 (error 108)
Metrics: {"server-negotiator.queue_time_us":204,"thread_start_us":90,"threads_started":1}
Aug 07 00:02:36 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:36 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:36.486479 13422 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:36.480526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:38405)
0807 00:02:36.480841 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:36.480856 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:36.480977 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:36.481249 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:36.481251 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:36.481466 (+ 215us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:36.481591 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:36.481598 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.482598 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.482602 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:36.483166 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:36.483174 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.483678 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.483682 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:36.483766 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:36.484298 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:36.484311 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:36.486311 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":241,"thread_start_us":92,"threads_started":1}
W20260807 00:02:36.486599 13420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:36.480673 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:38405 (local address 127.30.204.126:43273)
0807 00:02:36.480830 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:36.480835 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:36.480927 (+ 92us) server_negotiation.cc:407] Connection header received
0807 00:02:36.481086 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:36.481089 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:36.481137 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:36.481206 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:36.481777 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.482492 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.483280 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.483561 (+ 281us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.483651 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:36.486492 (+ 2841us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:38405: BlockingRecv error: recv got EOF from 127.30.204.66:38405 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 00:02:36 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:36 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:36.686087 13423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:36.679961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:53131)
0807 00:02:36.680242 (+ 281us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:36.680257 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:36.680371 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:36.680663 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:36.680665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:36.680853 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:36.680958 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:36.680964 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.681908 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.681911 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:36.682594 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:36.682608 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.683197 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.683201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:36.683277 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:36.683869 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:36.683882 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:36.685960 (+ 2078us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":89,"threads_started":1}
W20260807 00:02:36.686244 13420 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:36.680102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:53131 (local address 127.30.204.126:43273)
0807 00:02:36.680276 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:36.680280 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:36.680322 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:02:36.680457 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:36.680459 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:36.680517 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:36.680614 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:36.681100 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.681760 (+ 660us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.682748 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:36.683055 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:36.683147 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:36.686116 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:53131: BlockingRecv error: recv got EOF from 127.30.204.67:53131 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
Aug 07 00:02:37 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:37 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:37.309084 13424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:37.298711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:36557)
0807 00:02:37.299008 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:37.299023 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:37.299121 (+ 98us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:37.299378 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:37.299380 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:37.299608 (+ 228us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:37.299720 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:37.299727 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.300528 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.300531 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:37.301116 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:37.301126 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.301587 (+ 461us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.301590 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:37.301666 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:37.306618 (+ 4952us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:37.306634 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:37.308951 (+ 2317us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":214,"thread_start_us":91,"threads_started":1}
W20260807 00:02:37.309302 13425 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:37.298803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:36557 (local address 127.30.204.126:43273)
0807 00:02:37.299068 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:37.299072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:37.299087 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:02:37.299215 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:37.299217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:37.299264 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:37.299349 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:37.299846 (+ 497us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.300404 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.301238 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.301471 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.301552 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:37.309157 (+ 7605us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:36557: BlockingRecv error: recv got EOF from 127.30.204.65:36557 (error 108)
Metrics: {"server-negotiator.queue_time_us":189,"thread_start_us":87,"threads_started":1}
Aug 07 00:02:37 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:37 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:37.493683 13427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:37.487555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:35021)
0807 00:02:37.487855 (+ 300us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:37.487869 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:37.487984 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:37.488315 (+ 331us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:37.488318 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:37.488506 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:37.488622 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:37.488628 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.489515 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.489518 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:37.490143 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:37.490155 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.490632 (+ 477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.490636 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:37.490721 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:37.491293 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:37.491307 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:37.493491 (+ 2184us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":103,"threads_started":1}
W20260807 00:02:37.493827 13425 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:37.487680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:35021 (local address 127.30.204.126:43273)
0807 00:02:37.487857 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:37.487861 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:37.487974 (+ 113us) server_negotiation.cc:407] Connection header received
0807 00:02:37.488091 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:37.488093 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:37.488160 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:37.488268 (+ 108us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:37.488756 (+ 488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.489393 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.490261 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.490511 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.490606 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:37.493716 (+ 3110us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:35021: BlockingRecv error: recv got EOF from 127.30.204.66:35021 (error 108)
Metrics: {"server-negotiator.queue_time_us":70}
Aug 07 00:02:37 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:37 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:37.692505 13433 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:37.686910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:42155)
0807 00:02:37.687208 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:37.687224 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:37.687337 (+ 113us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:37.687611 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:37.687614 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:37.687840 (+ 226us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:37.687941 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:37.687946 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.688708 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.688712 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:37.689271 (+ 559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:37.689279 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.689725 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.689729 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:37.689817 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:37.690378 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:37.690392 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:37.692344 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":91,"threads_started":1}
W20260807 00:02:37.692622 13425 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:37.687020 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:42155 (local address 127.30.204.126:43273)
0807 00:02:37.687181 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:37.687186 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:37.687293 (+ 107us) server_negotiation.cc:407] Connection header received
0807 00:02:37.687428 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:37.687432 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:37.687505 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:37.687573 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:37.688056 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.688595 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.689382 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:37.689612 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:37.689705 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:37.692514 (+ 2809us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:42155: BlockingRecv error: recv got EOF from 127.30.204.67:42155 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Aug 07 00:02:38 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:38 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:38.315855 13434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:38.309936 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:36381)
0807 00:02:38.310350 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:38.310368 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:38.310495 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:38.310763 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:38.310767 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:38.310978 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:38.311079 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:38.311085 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.311898 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.311902 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:38.312473 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:38.312482 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.312937 (+ 455us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.312941 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:38.313028 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:38.313588 (+ 560us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:38.313602 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:38.315663 (+ 2061us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":313,"thread_start_us":87,"threads_started":1}
W20260807 00:02:38.316021 13435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:38.310051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:36381 (local address 127.30.204.126:43273)
0807 00:02:38.310316 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:38.310319 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:38.310481 (+ 162us) server_negotiation.cc:407] Connection header received
0807 00:02:38.310590 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:38.310593 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:38.310639 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:38.310714 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:38.311196 (+ 482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.311784 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.312585 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.312824 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.312913 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:38.315883 (+ 2970us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:36381: BlockingRecv error: recv got EOF from 127.30.204.65:36381 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":58,"threads_started":1}
Aug 07 00:02:38 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:38 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:38.500114 13436 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:38.494705 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:58271)
0807 00:02:38.494992 (+ 287us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:38.495005 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:38.495109 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:38.495349 (+ 240us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:38.495351 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:38.495565 (+ 214us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:38.495693 (+ 128us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:38.495699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.496418 (+ 719us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.496421 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:38.497032 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:38.497042 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.497533 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.497537 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:38.497612 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:38.498123 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:38.498133 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:38.499983 (+ 1850us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":206,"thread_start_us":84,"threads_started":1}
W20260807 00:02:38.500296 13435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:38.494760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:58271 (local address 127.30.204.126:43273)
0807 00:02:38.494911 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:38.494915 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:38.495074 (+ 159us) server_negotiation.cc:407] Connection header received
0807 00:02:38.495186 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:38.495188 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:38.495236 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:38.495303 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:38.495782 (+ 479us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.496312 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.497157 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.497391 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.497476 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:38.500161 (+ 2685us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:58271: BlockingRecv error: recv got EOF from 127.30.204.66:58271 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
Aug 07 00:02:38 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:38 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:38.699615 13437 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:38.693545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:43367)
0807 00:02:38.693949 (+ 404us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:38.693969 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:38.694118 (+ 149us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:38.694386 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:38.694389 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:38.694639 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:38.694748 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:38.694754 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.695601 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.695605 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:38.696182 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:38.696193 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.696667 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.696671 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:38.696756 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:38.697345 (+ 589us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:38.697358 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:38.699395 (+ 2037us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":144,"threads_started":1}
W20260807 00:02:38.699797 13435 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:38.693794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:43367 (local address 127.30.204.126:43273)
0807 00:02:38.693973 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:38.693977 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:38.694052 (+ 75us) server_negotiation.cc:407] Connection header received
0807 00:02:38.694217 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:38.694220 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:38.694271 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:38.694342 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:38.694878 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.695443 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.696315 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:38.696546 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:38.696641 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:38.699626 (+ 2985us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:43367: BlockingRecv error: recv got EOF from 127.30.204.67:43367 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Aug 07 00:02:39 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:39 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:39.323086 13440 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:39.316864 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:38287)
0807 00:02:39.317167 (+ 303us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:39.317181 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:39.317292 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:39.317664 (+ 372us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:39.317667 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:39.317862 (+ 195us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:39.317972 (+ 110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:39.317980 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.318913 (+ 933us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.318916 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:39.319581 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:39.319594 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.320149 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.320152 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:39.320235 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:39.320799 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:39.320815 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:39.322929 (+ 2114us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":84,"threads_started":1}
W20260807 00:02:39.323514 13444 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:39.317004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:38287 (local address 127.30.204.126:43273)
0807 00:02:39.317319 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:39.317323 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:39.317375 (+ 52us) server_negotiation.cc:407] Connection header received
0807 00:02:39.317424 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:39.317426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:39.317473 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:39.317553 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:39.318178 (+ 625us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.318771 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.319734 (+ 963us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.319994 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.320093 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:39.323341 (+ 3248us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:38287: BlockingRecv error: recv got EOF from 127.30.204.65:38287 (error 108)
Metrics: {"server-negotiator.queue_time_us":237,"thread_start_us":124,"threads_started":1}
Aug 07 00:02:39 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:39 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:39.507025 13445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:39.501081 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:43439)
0807 00:02:39.501389 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:39.501403 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:39.501510 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:39.501854 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:39.501857 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:39.502054 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:39.502161 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:39.502167 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.502942 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.502945 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:39.503666 (+ 721us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:39.503678 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.504168 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.504171 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:39.504245 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:39.504789 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:39.504802 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:39.506850 (+ 2048us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":101,"threads_started":1}
W20260807 00:02:39.507162 13444 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:39.501220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:43439 (local address 127.30.204.126:43273)
0807 00:02:39.501375 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:39.501380 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:39.501469 (+ 89us) server_negotiation.cc:407] Connection header received
0807 00:02:39.501655 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:39.501659 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:39.501718 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:39.501801 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:39.502279 (+ 478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.502822 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.503798 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.504029 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.504116 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:39.507046 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:43439: BlockingRecv error: recv got EOF from 127.30.204.66:43439 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
Aug 07 00:02:39 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:39 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:39.706447 13446 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:39.700595 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:34889)
0807 00:02:39.700914 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:39.700930 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:39.701078 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:39.701352 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:39.701355 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:39.701567 (+ 212us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:39.701681 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:39.701687 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.702519 (+ 832us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.702523 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:39.703190 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:39.703204 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.703767 (+ 563us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.703771 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:39.703908 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:39.704424 (+ 516us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:39.704437 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:39.706307 (+ 1870us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":246,"thread_start_us":118,"threads_started":1}
W20260807 00:02:39.706583 13444 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:39.700735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:34889 (local address 127.30.204.126:43273)
0807 00:02:39.700908 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:39.700913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:39.701010 (+ 97us) server_negotiation.cc:407] Connection header received
0807 00:02:39.701177 (+ 167us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:39.701180 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:39.701236 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:39.701309 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:39.701810 (+ 501us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.702365 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.703309 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:39.703632 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:39.703736 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:39.706469 (+ 2733us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:34889: BlockingRecv error: recv got EOF from 127.30.204.67:34889 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Aug 07 00:02:40 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:40 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:40.330061 13448 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:40.324143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:54093)
0807 00:02:40.324441 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:40.324455 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:40.324650 (+ 195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:40.324979 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:40.324983 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:40.325236 (+ 253us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:40.325359 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:40.325368 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.326375 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.326379 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:40.326995 (+ 616us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:40.327006 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.327543 (+ 537us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.327547 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:40.327623 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:40.328148 (+ 525us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:40.328161 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:40.329933 (+ 1772us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":81,"threads_started":1}
W20260807 00:02:40.330256 13449 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:40.324292 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:54093 (local address 127.30.204.126:43273)
0807 00:02:40.324544 (+ 252us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:40.324547 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:40.324564 (+ 17us) server_negotiation.cc:407] Connection header received
0807 00:02:40.324749 (+ 185us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:40.324751 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:40.324818 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:40.324931 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:40.325509 (+ 578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.326237 (+ 728us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.327131 (+ 894us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.327381 (+ 250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.327515 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:40.330102 (+ 2587us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:54093: BlockingRecv error: recv got EOF from 127.30.204.65:54093 (error 108)
Metrics: {"server-negotiator.queue_time_us":176,"thread_start_us":74,"threads_started":1}
Aug 07 00:02:40 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:40 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:40.514145 13450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:40.508048 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:39107)
0807 00:02:40.508337 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:40.508352 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:40.508461 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:40.508748 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:40.508751 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:40.508939 (+ 188us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:40.509078 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:40.509087 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.510197 (+ 1110us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.510201 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:40.510933 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:40.510947 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.511437 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.511440 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:40.511565 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:40.512122 (+ 557us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:40.512137 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:40.514003 (+ 1866us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":82,"threads_started":1}
W20260807 00:02:40.514349 13449 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:40.508174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:39107 (local address 127.30.204.126:43273)
0807 00:02:40.508341 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:40.508345 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:40.508421 (+ 76us) server_negotiation.cc:407] Connection header received
0807 00:02:40.508549 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:40.508552 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:40.508601 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:40.508691 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:40.509280 (+ 589us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.510021 (+ 741us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.511066 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.511301 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.511386 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:40.514212 (+ 2826us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:39107: BlockingRecv error: recv got EOF from 127.30.204.66:39107 (error 108)
Metrics: {"server-negotiator.queue_time_us":78}
Aug 07 00:02:40 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:40 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:40.713627 13451 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:40.707378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:40507)
0807 00:02:40.707724 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:40.707738 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:40.707850 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:40.708140 (+ 290us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:40.708142 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:40.708374 (+ 232us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:40.708483 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:40.708489 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.709323 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.709326 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:40.709910 (+ 584us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:40.709920 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.710507 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.710512 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:40.710597 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:40.711178 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:40.711191 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:40.713491 (+ 2300us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":116,"threads_started":1}
W20260807 00:02:40.713874 13449 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:40.707555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:40507 (local address 127.30.204.126:43273)
0807 00:02:40.707709 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:40.707713 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:40.707841 (+ 128us) server_negotiation.cc:407] Connection header received
0807 00:02:40.707921 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:40.707923 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:40.707977 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:40.708055 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:40.708630 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.709190 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.710064 (+ 874us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:40.710327 (+ 263us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:40.710422 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:40.713729 (+ 3307us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:40507: BlockingRecv error: recv got EOF from 127.30.204.67:40507 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
Aug 07 00:02:41 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:41 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:41.336748 13453 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:41.331102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:54515)
0807 00:02:41.331376 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:41.331390 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:41.331535 (+ 145us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:41.331806 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:41.331809 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:41.332012 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:41.332113 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:41.332120 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.332897 (+ 777us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.332901 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:41.333471 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:41.333480 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.333949 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.333954 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:41.334040 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:41.334580 (+ 540us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:41.334593 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:41.336593 (+ 2000us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":79,"threads_started":1}
W20260807 00:02:41.336961 13452 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:41.331111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:54515 (local address 127.30.204.126:43273)
0807 00:02:41.331372 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:41.331377 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:41.331488 (+ 111us) server_negotiation.cc:407] Connection header received
0807 00:02:41.331624 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:41.331627 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:41.331677 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:41.331762 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:41.332230 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.332784 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.333586 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.333832 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.333925 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:41.336802 (+ 2877us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:54515: BlockingRecv error: recv got EOF from 127.30.204.65:54515 (error 108)
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":76,"threads_started":1}
Aug 07 00:02:41 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:41 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:41.520900 13454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:41.514992 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:33367)
0807 00:02:41.515274 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:41.515291 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:41.515419 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:41.515779 (+ 360us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:41.515783 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:41.516007 (+ 224us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:41.516139 (+ 132us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:41.516148 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.516975 (+ 827us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.516978 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:41.517584 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:41.517594 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.518099 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.518102 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:41.518172 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:41.518804 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:41.518818 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:41.520746 (+ 1928us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":90,"threads_started":1}
W20260807 00:02:41.521025 13452 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:41.515132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:33367 (local address 127.30.204.126:43273)
0807 00:02:41.515278 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:41.515283 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:41.515365 (+ 82us) server_negotiation.cc:407] Connection header received
0807 00:02:41.515558 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:41.515561 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:41.515623 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:41.515711 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:41.516282 (+ 571us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.516847 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.517728 (+ 881us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.517966 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.518061 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:41.520913 (+ 2852us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:33367: BlockingRecv error: recv got EOF from 127.30.204.66:33367 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
Aug 07 00:02:41 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:41 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
W20260807 00:02:41.720152 13452 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:41.714709 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:49595 (local address 127.30.204.126:43273)
0807 00:02:41.714900 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:41.714905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:41.714966 (+ 61us) server_negotiation.cc:407] Connection header received
0807 00:02:41.715105 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:41.715107 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:41.715159 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:41.715231 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:41.715701 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.716252 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.717003 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.717224 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.717296 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:41.720040 (+ 2744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:49595: BlockingRecv error: recv got EOF from 127.30.204.67:49595 (error 108)
Metrics: {"server-negotiator.queue_time_us":85}
I20260807 00:02:41.720481 13457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:41.714591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:49595)
0807 00:02:41.714892 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:41.714908 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:41.715022 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:41.715269 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:41.715271 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:41.715494 (+ 223us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:41.715601 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:41.715607 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.716349 (+ 742us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.716352 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:41.716915 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:41.716922 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:41.717316 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:41.717319 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:41.717380 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:41.717948 (+ 568us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:41.717961 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:41.720341 (+ 2380us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":96,"threads_started":1}
Aug 07 00:02:42 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:42 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:42.347935 13461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:42.337667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:43455)
0807 00:02:42.337984 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:42.337999 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:42.338108 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:42.338438 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:42.338441 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:42.338635 (+ 194us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:42.338760 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:42.338766 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.339666 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.339669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:42.340254 (+ 585us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:42.340265 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.340780 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.340784 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:42.340876 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:42.345572 (+ 4696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:42.345590 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:42.347793 (+ 2203us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":226,"thread_start_us":102,"threads_started":1}
W20260807 00:02:42.348212 13462 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:42.337830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:43455 (local address 127.30.204.126:43273)
0807 00:02:42.338157 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:42.338161 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:42.338177 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:42.338221 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:42.338223 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:42.338268 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:42.338351 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:42.338932 (+ 581us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.339571 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.340375 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.340618 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.340706 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:42.348054 (+ 7348us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:43455: BlockingRecv error: recv got EOF from 127.30.204.65:43455 (error 108)
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":66,"threads_started":1}
Aug 07 00:02:42 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:42 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:42.527612 13463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:42.521989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:43703)
0807 00:02:42.522296 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:42.522312 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:42.522427 (+ 115us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:42.522757 (+ 330us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:42.522761 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:42.522967 (+ 206us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:42.523094 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:42.523103 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.523905 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.523908 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:42.524527 (+ 619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:42.524540 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.525020 (+ 480us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.525023 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:42.525122 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:42.525602 (+ 480us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:42.525614 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:42.527444 (+ 1830us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":108,"threads_started":1}
W20260807 00:02:42.527768 13462 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:42.522102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:43703 (local address 127.30.204.126:43273)
0807 00:02:42.522267 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:42.522271 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:42.522383 (+ 112us) server_negotiation.cc:407] Connection header received
0807 00:02:42.522567 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:42.522577 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:42.522632 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:42.522705 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:42.523204 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.523786 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.524634 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.524911 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.524994 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:42.527637 (+ 2643us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:43703: BlockingRecv error: recv got EOF from 127.30.204.66:43703 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
Aug 07 00:02:42 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:42 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.67@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:42.726975 13464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:42.721223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.67:40957)
0807 00:02:42.721554 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:42.721567 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:42.721685 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:42.721951 (+ 266us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:42.721953 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:42.722192 (+ 239us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:42.722295 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:42.722301 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.723120 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.723124 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:42.723768 (+ 644us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:42.723779 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.724243 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.724247 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:42.724336 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:42.724868 (+ 532us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:42.724881 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:42.726804 (+ 1923us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":95,"threads_started":1}
W20260807 00:02:42.727098 13462 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:42.721340 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:40957 (local address 127.30.204.126:43273)
0807 00:02:42.721480 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:42.721485 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:42.721642 (+ 157us) server_negotiation.cc:407] Connection header received
0807 00:02:42.721775 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:42.721779 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:42.721833 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:42.721916 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:42.722418 (+ 502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.722994 (+ 576us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.723889 (+ 895us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:42.724125 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:42.724227 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:42.726990 (+ 2763us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.67:40957: BlockingRecv error: recv got EOF from 127.30.204.67:40957 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Aug 07 00:02:43 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:43 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.65@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:43.355028 13465 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:43.349090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.65:35855)
0807 00:02:43.349449 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:43.349464 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:43.349591 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:43.349864 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:43.349867 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:43.350053 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:43.350155 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:43.350161 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.350922 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.350926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:43.351522 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:43.351531 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.352037 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.352040 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:43.352113 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:43.352736 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:43.352751 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:43.354884 (+ 2133us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":269,"thread_start_us":127,"threads_started":1}
W20260807 00:02:43.355207 13466 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:43.349259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:35855 (local address 127.30.204.126:43273)
0807 00:02:43.349520 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:43.349524 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:43.349540 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:43.349679 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:43.349682 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:43.349729 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:43.349812 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:43.350263 (+ 451us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.350807 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.351660 (+ 853us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.351912 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.351996 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:43.355065 (+ 3069us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.65:35855: BlockingRecv error: recv got EOF from 127.30.204.65:35855 (error 108)
Metrics: {"server-negotiator.queue_time_us":186,"thread_start_us":63,"threads_started":1}
Aug 07 00:02:43 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
Aug 07 00:02:43 dist-test-slave-pww0 krb5kdc[11804](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.30.204.66@KRBTEST.COM for oryx/127.30.204.126@KRBTEST.COM, Server not found in Kerberos database
I20260807 00:02:43.534484 13468 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:43.528604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43273 (local address 127.30.204.66:37599)
0807 00:02:43.528918 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:43.528934 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:43.529066 (+ 132us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:43.529343 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:43.529346 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:43.529634 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:43.529771 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:43.529780 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.530693 (+ 913us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.530696 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:43.531383 (+ 687us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:43.531395 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.531906 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.531910 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:43.531983 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:43.532483 (+ 500us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:43.532495 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:43.534355 (+ 1860us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.30.204.126:43273: Server oryx/127.30.204.126@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":95,"threads_started":1}
W20260807 00:02:43.534739 13466 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:43.528711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:37599 (local address 127.30.204.126:43273)
0807 00:02:43.528903 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:43.528907 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:43.529022 (+ 115us) server_negotiation.cc:407] Connection header received
0807 00:02:43.529166 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:43.529168 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:43.529228 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:43.529317 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:43.529932 (+ 615us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.530568 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.531561 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.531804 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.531890 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:43.534609 (+ 2719us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.30.204.66:37599: BlockingRecv error: recv got EOF from 127.30.204.66:37599 (error 108)
Metrics: {"server-negotiator.queue_time_us":71}
I20260807 00:02:43.618820 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 12722
I20260807 00:02:43.649686 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 12949
I20260807 00:02:43.670094 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 13161
I20260807 00:02:43.681880 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 12615
2026-08-07T00:02:43Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (18434 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-08-07T00:02:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:43Z Disabled control of system clock
I20260807 00:02:43.718109 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34349
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:34349
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260807 00:02:43.836700 13475 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:43.837013 13475 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:43.837049 13475 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:43.841305 13475 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:43.841398 13475 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:43.841413 13475 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:43.841428 13475 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:43.841442 13475 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:43.846565 13475 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:34349
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:34349
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13475
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:43.847774 13475 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:43.849212 13475 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:43.858656 13489 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:43.858657 13486 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:43.858656 13485 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:43.858951 13475 server_base.cc:1034] running on GCE node
I20260807 00:02:43.859632 13475 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:43.860904 13475 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:43.862097 13475 hybrid_clock.cc:648] HybridClock initialized: now 1786060963862079 us; error 36 us; skew 500 ppm
I20260807 00:02:43.864248 13475 webserver.cc:460] Webserver started at http://127.30.204.126:36629/ using document root <none> and password file <none>
I20260807 00:02:43.864882 13475 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:43.864953 13475 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:43.865136 13475 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:43.867406 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "51993a29517b4c168685d66f87b0cb12"
format_stamp: "Formatted at 2026-08-07 00:02:43 on dist-test-slave-pww0"
server_key: "1ca21fbba15b3097ef2991d56c5d371b"
server_key_iv: "2726fd52da12adf952cf5e4854281219"
server_key_version: "encryptionkey@0"
I20260807 00:02:43.868085 13475 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "51993a29517b4c168685d66f87b0cb12"
format_stamp: "Formatted at 2026-08-07 00:02:43 on dist-test-slave-pww0"
server_key: "1ca21fbba15b3097ef2991d56c5d371b"
server_key_iv: "2726fd52da12adf952cf5e4854281219"
server_key_version: "encryptionkey@0"
I20260807 00:02:43.872421 13475 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:02:43.875298 13497 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:43.876911 13475 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:43.877076 13475 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "51993a29517b4c168685d66f87b0cb12"
format_stamp: "Formatted at 2026-08-07 00:02:43 on dist-test-slave-pww0"
server_key: "1ca21fbba15b3097ef2991d56c5d371b"
server_key_iv: "2726fd52da12adf952cf5e4854281219"
server_key_version: "encryptionkey@0"
I20260807 00:02:43.877213 13475 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:43.908512 13475 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:43.909332 13475 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:43.909533 13475 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:43.917050 13475 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:34349
I20260807 00:02:43.917071 13579 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:34349 every 8 connection(s)
I20260807 00:02:43.918076 13475 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:43.921540 13582 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:43.925640 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 13475
I20260807 00:02:43.925773 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:43.926044 31537 external_mini_cluster.cc:1428] Setting key 368835918b711abdc503bbff46771d31
I20260807 00:02:43.928145 13582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12: Bootstrap starting.
I20260807 00:02:43.931192 13582 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:43.932219 13582 log.cc:824] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:43.934782 13582 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12: No bootstrap required, opened a new log
I20260807 00:02:43.937770 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:43.927681 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41746 (local address 127.30.204.126:34349)
0807 00:02:43.928468 (+ 787us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:43.928489 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:43.928539 (+ 50us) server_negotiation.cc:407] Connection header received
0807 00:02:43.929518 (+ 979us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:43.929544 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:43.930099 (+ 555us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:43.930529 (+ 430us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:43.931018 (+ 489us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.932548 (+ 1530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.933712 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:43.934422 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:43.934591 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:43.935426 (+ 835us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:43.935433 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:43.935476 (+ 43us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:43.936023 (+ 547us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:43.936107 (+ 84us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:43.936134 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:43.936251 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:43.936419 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:02:43.936720 (+ 301us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":320,"thread_start_us":132,"threads_started":1}
I20260807 00:02:43.939780 13582 raft_consensus.cc:348] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51993a29517b4c168685d66f87b0cb12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34349 } }
I20260807 00:02:43.940088 13582 raft_consensus.cc:374] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:43.940182 13582 raft_consensus.cc:729] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 51993a29517b4c168685d66f87b0cb12, State: Initialized, Role: FOLLOWER
I20260807 00:02:43.940661 13582 consensus_queue.cc:260] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51993a29517b4c168685d66f87b0cb12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34349 } }
I20260807 00:02:43.940809 13582 raft_consensus.cc:388] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:43.940877 13582 raft_consensus.cc:482] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:43.940979 13582 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:43.942024 13582 raft_consensus.cc:504] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51993a29517b4c168685d66f87b0cb12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34349 } }
I20260807 00:02:43.942395 13582 leader_election.cc:302] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 51993a29517b4c168685d66f87b0cb12; no voters:
I20260807 00:02:43.942746 13582 leader_election.cc:288] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:43.942875 13591 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:43.943178 13591 raft_consensus.cc:686] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [term 1 LEADER]: Becoming Leader. State: Replica: 51993a29517b4c168685d66f87b0cb12, State: Running, Role: LEADER
I20260807 00:02:43.943569 13591 consensus_queue.cc:237] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51993a29517b4c168685d66f87b0cb12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34349 } }
I20260807 00:02:43.944183 13582 sys_catalog.cc:564] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:43.945369 13592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "51993a29517b4c168685d66f87b0cb12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51993a29517b4c168685d66f87b0cb12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34349 } } }
I20260807 00:02:43.945519 13592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:43.945849 13593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 51993a29517b4c168685d66f87b0cb12. Latest consensus state: current_term: 1 leader_uuid: "51993a29517b4c168685d66f87b0cb12" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "51993a29517b4c168685d66f87b0cb12" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 34349 } } }
I20260807 00:02:43.945923 13605 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:43.946030 13593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:43.950006 13605 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:43.956518 13605 catalog_manager.cc:1347] Generated new cluster ID: 2980253fb75a4dff9ad84663873fce24
I20260807 00:02:43.956611 13605 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:43.974859 13605 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:43.975951 13605 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:43.984474 13605 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 51993a29517b4c168685d66f87b0cb12: Generated new TSK 0
I20260807 00:02:43.985277 13605 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:02:43.996062 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34349
--builtin_ntp_servers=127.30.204.84:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260807 00:02:44.131744 13621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:44.132061 13621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:44.132118 13621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:44.136384 13621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:44.136457 13621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:44.136535 13621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:44.141602 13621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:44.142884 13621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:44.144521 13621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:44.155378 13628 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:44.155428 13629 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:44.155378 13631 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:44.155920 13621 server_base.cc:1034] running on GCE node
I20260807 00:02:44.156432 13621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:44.157164 13621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:44.158443 13621 hybrid_clock.cc:648] HybridClock initialized: now 1786060964158417 us; error 39 us; skew 500 ppm
I20260807 00:02:44.160784 13621 webserver.cc:460] Webserver started at http://127.30.204.65:41359/ using document root <none> and password file <none>
I20260807 00:02:44.161439 13621 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:44.161521 13621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:44.161744 13621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:44.164252 13621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "5570048bc13f497a90883812d49fe477"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "8454cc0c79a8a5907fcf6ea3ff1825db"
server_key_iv: "29256958633a4e11f386fca2ea2fb47b"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.164898 13621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "5570048bc13f497a90883812d49fe477"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "8454cc0c79a8a5907fcf6ea3ff1825db"
server_key_iv: "29256958633a4e11f386fca2ea2fb47b"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.169116 13621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:02:44.172209 13639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.173566 13621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:02:44.173703 13621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "5570048bc13f497a90883812d49fe477"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "8454cc0c79a8a5907fcf6ea3ff1825db"
server_key_iv: "29256958633a4e11f386fca2ea2fb47b"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.173828 13621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:44.191222 13621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:44.192066 13621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:44.192271 13621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:44.192952 13621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:44.194034 13621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:44.194084 13621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.194145 13621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:44.194175 13621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.203225 13621 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:46831
I20260807 00:02:44.203261 13809 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:46831 every 8 connection(s)
I20260807 00:02:44.204303 13621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:44.205134 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 13621
I20260807 00:02:44.205238 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:44.205569 31537 external_mini_cluster.cc:1428] Setting key ae7ee62653828fba55e54489d5320ff1
I20260807 00:02:44.207881 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34349
--builtin_ntp_servers=127.30.204.84:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 00:02:44.215062 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:44.206371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:40283 (local address 127.30.204.126:34349)
0807 00:02:44.206490 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:44.206494 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:44.208040 (+ 1546us) server_negotiation.cc:407] Connection header received
0807 00:02:44.209746 (+ 1706us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:44.209749 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:44.209802 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:44.209872 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:44.211189 (+ 1317us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.211786 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.213031 (+ 1245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.213261 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.213357 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:44.214571 (+ 1214us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:44.214574 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:44.214576 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:44.214590 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:44.214610 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:44.214619 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:44.214678 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:44.214827 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 00:02:44.214901 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:02:44.216070 13815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:44.206993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34349 (local address 127.30.204.65:40283)
0807 00:02:44.207865 (+ 872us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:44.207931 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:44.209469 (+ 1538us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:44.210115 (+ 646us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:44.210126 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:44.210256 (+ 130us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:44.211014 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:44.211049 (+ 35us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.211923 (+ 874us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.211926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:44.212899 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:44.212911 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.213425 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.213430 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:44.213601 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:44.214285 (+ 684us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:44.214287 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:44.214334 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:44.214411 (+ 77us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:44.214688 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:44.214699 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:44.214705 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:02:44.214822 (+ 117us) client_negotiation.cc:240] Negotiation successful
0807 00:02:44.215152 (+ 330us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":160,"threads_started":1}
I20260807 00:02:44.217491 13810 heartbeater.cc:347] Connected to a master server at 127.30.204.126:34349
I20260807 00:02:44.217756 13810 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:44.218324 13810 heartbeater.cc:511] Master 127.30.204.126:34349 requested a full tablet report, sending...
I20260807 00:02:44.220238 13522 ts_manager.cc:194] Registered new tserver with Master: 5570048bc13f497a90883812d49fe477 (127.30.204.65:46831)
I20260807 00:02:44.221921 13522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.65:40283
W20260807 00:02:44.329888 13816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:44.330216 13816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:44.330289 13816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:44.334605 13816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:44.334693 13816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:44.334774 13816 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:44.339990 13816 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:44.341212 13816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:44.342729 13816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:44.353199 13821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:44.353226 13825 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:44.353226 13832 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:44.353338 13816 server_base.cc:1034] running on GCE node
I20260807 00:02:44.354069 13816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:44.354839 13816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:44.356042 13816 hybrid_clock.cc:648] HybridClock initialized: now 1786060964356016 us; error 39 us; skew 500 ppm
I20260807 00:02:44.358188 13816 webserver.cc:460] Webserver started at http://127.30.204.66:41077/ using document root <none> and password file <none>
I20260807 00:02:44.358857 13816 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:44.358940 13816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:44.359165 13816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:44.361433 13816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "220eba01bf444d5092c063ecf40e7eda"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "92cd76fadd4b071828f01cf42ee928be"
server_key_iv: "f4e905643f7595772343c35ab8c72afc"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.362026 13816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "220eba01bf444d5092c063ecf40e7eda"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "92cd76fadd4b071828f01cf42ee928be"
server_key_iv: "f4e905643f7595772343c35ab8c72afc"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.366108 13816 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:44.369197 13842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.370872 13816 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:44.371042 13816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "220eba01bf444d5092c063ecf40e7eda"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "92cd76fadd4b071828f01cf42ee928be"
server_key_iv: "f4e905643f7595772343c35ab8c72afc"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.371179 13816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:44.385667 13816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:44.386504 13816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:44.386716 13816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:44.387382 13816 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:44.388469 13816 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:44.388520 13816 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.388577 13816 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:44.388602 13816 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.397703 13816 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:39795
I20260807 00:02:44.397794 14017 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:39795 every 8 connection(s)
I20260807 00:02:44.398715 13816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:44.406482 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 13816
I20260807 00:02:44.406592 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:44.406909 31537 external_mini_cluster.cc:1428] Setting key b8e75cd0f7612d3202da36de04c30294
I20260807 00:02:44.408699 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:44.400629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:52821 (local address 127.30.204.126:34349)
0807 00:02:44.400763 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:44.400767 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:44.401944 (+ 1177us) server_negotiation.cc:407] Connection header received
0807 00:02:44.403359 (+ 1415us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:44.403363 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:44.403428 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:44.403518 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:44.404520 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.405059 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.406402 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.406722 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.406832 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:44.408185 (+ 1353us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:44.408189 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:44.408192 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:44.408206 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:44.408228 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:44.408237 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:44.408299 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:44.408481 (+ 182us) server_negotiation.cc:299] Negotiation successful
0807 00:02:44.408540 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:02:44.409227 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34349
--builtin_ntp_servers=127.30.204.84:42961
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260807 00:02:44.409827 14021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:44.401037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34349 (local address 127.30.204.66:52821)
0807 00:02:44.401774 (+ 737us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:44.401820 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:44.403123 (+ 1303us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:44.403730 (+ 607us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:44.403737 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:44.403831 (+ 94us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:44.404362 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:44.404377 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.405252 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.405255 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:44.406239 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:44.406252 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.406914 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.406919 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:44.407152 (+ 233us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:44.407862 (+ 710us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:44.407863 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:44.407916 (+ 53us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:44.407998 (+ 82us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:44.408312 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:44.408324 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:44.408330 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:02:44.408475 (+ 145us) client_negotiation.cc:240] Negotiation successful
0807 00:02:44.408822 (+ 347us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":129,"threads_started":1}
I20260807 00:02:44.411072 14019 heartbeater.cc:347] Connected to a master server at 127.30.204.126:34349
I20260807 00:02:44.411319 14019 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:44.411939 14019 heartbeater.cc:511] Master 127.30.204.126:34349 requested a full tablet report, sending...
I20260807 00:02:44.413096 13522 ts_manager.cc:194] Registered new tserver with Master: 220eba01bf444d5092c063ecf40e7eda (127.30.204.66:39795)
I20260807 00:02:44.413883 13522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.66:52821
W20260807 00:02:44.527900 14022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:44.528193 14022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:44.528231 14022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:44.532384 14022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:44.532457 14022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:44.532511 14022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:44.537868 14022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42961
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:34349
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:44.539045 14022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:44.540514 14022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:44.550874 14033 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:44.551038 14022 server_base.cc:1034] running on GCE node
W20260807 00:02:44.550915 14030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:44.550931 14028 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:44.551672 14022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:44.552366 14022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:44.553553 14022 hybrid_clock.cc:648] HybridClock initialized: now 1786060964553533 us; error 35 us; skew 500 ppm
I20260807 00:02:44.555660 14022 webserver.cc:460] Webserver started at http://127.30.204.67:45695/ using document root <none> and password file <none>
I20260807 00:02:44.556269 14022 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:44.556346 14022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:44.556574 14022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:44.558832 14022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "5afbca33094a4c8f9e834f7b374ebe2e"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "f23bc70d256cd6b4e5a8033e66710e94"
server_key_iv: "3fca21d407c4993d84631ec3a184c537"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.559648 14022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "5afbca33094a4c8f9e834f7b374ebe2e"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "f23bc70d256cd6b4e5a8033e66710e94"
server_key_iv: "3fca21d407c4993d84631ec3a184c537"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.563805 14022 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:44.566546 14041 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.568010 14022 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:44.568150 14022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "5afbca33094a4c8f9e834f7b374ebe2e"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "f23bc70d256cd6b4e5a8033e66710e94"
server_key_iv: "3fca21d407c4993d84631ec3a184c537"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.568274 14022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:44.580022 14022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:44.580780 14022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:44.580956 14022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:44.581573 14022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:44.582618 14022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:44.582671 14022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.582738 14022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:44.582764 14022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.592106 14022 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:40437
I20260807 00:02:44.592149 14226 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:40437 every 8 connection(s)
I20260807 00:02:44.593058 14022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:44.597396 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 14022
I20260807 00:02:44.597522 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:44.597875 31537 external_mini_cluster.cc:1428] Setting key d811ed270f46fc9ecf8229144c5b24be
I20260807 00:02:44.602340 13589 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:44.594941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:60837 (local address 127.30.204.126:34349)
0807 00:02:44.595068 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:44.595072 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:44.596167 (+ 1095us) server_negotiation.cc:407] Connection header received
0807 00:02:44.597525 (+ 1358us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:44.597529 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:44.597595 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:44.597692 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:44.598693 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.599363 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.600545 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.600778 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.600866 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:44.601877 (+ 1011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:44.601881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:44.601884 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:44.601896 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:44.601917 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:44.601925 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:44.601981 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:44.602125 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 00:02:44.602187 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:02:44.603252 14230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:44.595287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:34349 (local address 127.30.204.67:60837)
0807 00:02:44.596002 (+ 715us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:44.596044 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:44.597292 (+ 1248us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:44.597853 (+ 561us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:44.597860 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:44.597948 (+ 88us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:44.598546 (+ 598us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:44.598563 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.599520 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.599523 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:44.600425 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:44.600435 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:44.600918 (+ 483us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:44.600923 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:44.601082 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:44.601604 (+ 522us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:44.601606 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:44.601646 (+ 40us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:44.601714 (+ 68us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:44.601991 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:44.602001 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:44.602007 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:02:44.602116 (+ 109us) client_negotiation.cc:240] Negotiation successful
0807 00:02:44.602410 (+ 294us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":151,"threads_started":1}
I20260807 00:02:44.604374 14227 heartbeater.cc:347] Connected to a master server at 127.30.204.126:34349
I20260807 00:02:44.604610 14227 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:44.605154 14227 heartbeater.cc:511] Master 127.30.204.126:34349 requested a full tablet report, sending...
I20260807 00:02:44.606209 13522 ts_manager.cc:194] Registered new tserver with Master: 5afbca33094a4c8f9e834f7b374ebe2e (127.30.204.67:40437)
I20260807 00:02:44.606774 13522 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.67:60837
I20260807 00:02:44.611891 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 00:02:44.614640 13589 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:44.613764 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41752 (local address 127.30.204.126:34349)
0807 00:02:44.613915 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:44.613919 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:44.614060 (+ 141us) server_negotiation.cc:407] Connection header received
0807 00:02:44.614167 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:44.614169 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:44.614212 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:44.614288 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:44.614498 (+ 210us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:41752: BlockingRecv error: recv got EOF from 127.0.0.1:41752 (error 108)
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 00:02:44.614732 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 13621
I20260807 00:02:44.641223 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 13816
I20260807 00:02:44.664510 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 14022
I20260807 00:02:44.698418 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 13475
2026-08-07T00:02:44Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (1020 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-08-07T00:02:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:44Z Disabled control of system clock
I20260807 00:02:44.738116 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:44921
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:44921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 00:02:44.857573 14243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:44.857909 14243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:44.857952 14243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:44.862262 14243 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:44.862362 14243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:44.862380 14243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:44.862401 14243 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:44.862418 14243 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:44.867522 14243 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:44921
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:44921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:44.868741 14243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:44.870247 14243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:44.879804 14254 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:44.879794 14258 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:44.879794 14253 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:44.879860 14243 server_base.cc:1034] running on GCE node
I20260807 00:02:44.880717 14243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:44.881937 14243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:44.883132 14243 hybrid_clock.cc:648] HybridClock initialized: now 1786060964883116 us; error 31 us; skew 500 ppm
I20260807 00:02:44.885475 14243 webserver.cc:460] Webserver started at http://127.30.204.126:46027/ using document root <none> and password file <none>
I20260807 00:02:44.886129 14243 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:44.886211 14243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:44.886448 14243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:44.888837 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "5308d498dc1d4b6d997e3b9929aa19b7"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "a1b89aeefc5cac8d7c6cae0510c1be6f"
server_key_iv: "a735903669107faeddc8800197dc7336"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.889472 14243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "5308d498dc1d4b6d997e3b9929aa19b7"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "a1b89aeefc5cac8d7c6cae0510c1be6f"
server_key_iv: "a735903669107faeddc8800197dc7336"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.893745 14243 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:44.896584 14267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:44.897997 14243 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:44.898134 14243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "5308d498dc1d4b6d997e3b9929aa19b7"
format_stamp: "Formatted at 2026-08-07 00:02:44 on dist-test-slave-pww0"
server_key: "a1b89aeefc5cac8d7c6cae0510c1be6f"
server_key_iv: "a735903669107faeddc8800197dc7336"
server_key_version: "encryptionkey@0"
I20260807 00:02:44.898258 14243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:44.903172 14243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:44.903875 14243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:44.904045 14243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:44.911319 14243 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:44921
I20260807 00:02:44.911350 14338 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:44921 every 8 connection(s)
I20260807 00:02:44.912428 14243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:44.914608 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 14243
I20260807 00:02:44.914712 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:44.914994 31537 external_mini_cluster.cc:1428] Setting key 8b92b0c4d67686a75646842f3aeb9445
I20260807 00:02:44.916132 14339 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:44.921114 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:44.916566 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.30.204.126:44921)
0807 00:02:44.917347 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:44.917364 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:44.917406 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:02:44.918087 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:44.918104 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:44.918485 (+ 381us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:44.918829 (+ 344us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:44.919712 (+ 883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:44.919717 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:44.919730 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:44.919742 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:44.919803 (+ 61us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:44.919821 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:44.919972 (+ 151us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:44.920059 (+ 87us) server_negotiation.cc:299] Negotiation successful
0807 00:02:44.920321 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":158,"threads_started":1}
I20260807 00:02:44.922672 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7: Bootstrap starting.
I20260807 00:02:44.925172 14339 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:44.926023 14339 log.cc:824] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:44.928373 14339 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7: No bootstrap required, opened a new log
I20260807 00:02:44.932297 14339 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44921 } }
I20260807 00:02:44.932617 14339 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:44.932670 14339 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5308d498dc1d4b6d997e3b9929aa19b7, State: Initialized, Role: FOLLOWER
I20260807 00:02:44.933199 14339 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44921 } }
I20260807 00:02:44.933357 14339 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:44.933408 14339 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:44.933485 14339 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:44.934463 14339 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44921 } }
I20260807 00:02:44.934810 14339 leader_election.cc:302] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5308d498dc1d4b6d997e3b9929aa19b7; no voters:
I20260807 00:02:44.935120 14339 leader_election.cc:288] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:44.935251 14351 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:44.935500 14351 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [term 1 LEADER]: Becoming Leader. State: Replica: 5308d498dc1d4b6d997e3b9929aa19b7, State: Running, Role: LEADER
I20260807 00:02:44.935906 14351 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44921 } }
I20260807 00:02:44.936458 14339 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:44.937685 14352 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44921 } } }
I20260807 00:02:44.937829 14352 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:44.938256 14369 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:44.938200 14353 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5308d498dc1d4b6d997e3b9929aa19b7. Latest consensus state: current_term: 1 leader_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5308d498dc1d4b6d997e3b9929aa19b7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 44921 } } }
I20260807 00:02:44.938436 14353 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:44.941337 14369 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:44.948187 14369 catalog_manager.cc:1347] Generated new cluster ID: f19e921de2ea460fae24b108896a94f4
I20260807 00:02:44.948287 14369 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:44.964028 14369 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:44.964418 14369 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:44.969419 14369 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5308d498dc1d4b6d997e3b9929aa19b7: Generated new TSK 0
I20260807 00:02:44.970247 14369 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:02:44.976225 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44921
--builtin_ntp_servers=127.30.204.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 00:02:45.100993 14387 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:45.101326 14387 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:45.101372 14387 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:45.105715 14387 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:45.105791 14387 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:45.105850 14387 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:45.110916 14387 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14387
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:45.112172 14387 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:45.113735 14387 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:02:45.124353 14387 server_base.cc:1034] running on GCE node
W20260807 00:02:45.124362 14399 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:45.124521 14398 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:45.124362 14402 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:45.128074 14387 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:45.128959 14387 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:45.130177 14387 hybrid_clock.cc:648] HybridClock initialized: now 1786060965130154 us; error 51 us; skew 500 ppm
I20260807 00:02:45.132319 14387 webserver.cc:460] Webserver started at http://127.30.204.65:42107/ using document root <none> and password file <none>
I20260807 00:02:45.132934 14387 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:45.133005 14387 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:45.133234 14387 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:45.135560 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "e94bd43d7e5c41bc98c2274526acee73"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "6e3bd4a29f88f76c7633243203e63009"
server_key_iv: "d18b9bcaedc25d5a2c64ddc4ff946ad3"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.136216 14387 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "e94bd43d7e5c41bc98c2274526acee73"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "6e3bd4a29f88f76c7633243203e63009"
server_key_iv: "d18b9bcaedc25d5a2c64ddc4ff946ad3"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.140443 14387 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:45.143296 14411 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.144816 14387 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:02:45.144963 14387 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "e94bd43d7e5c41bc98c2274526acee73"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "6e3bd4a29f88f76c7633243203e63009"
server_key_iv: "d18b9bcaedc25d5a2c64ddc4ff946ad3"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.145105 14387 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:45.150154 14387 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:45.150883 14387 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:45.151080 14387 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:45.151736 14387 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:45.152766 14387 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:45.152817 14387 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.152855 14387 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:45.152873 14387 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.161970 14387 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:41391
I20260807 00:02:45.162025 14585 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:41391 every 8 connection(s)
I20260807 00:02:45.162992 14387 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:45.164415 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 14387
I20260807 00:02:45.164546 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:45.164935 31537 external_mini_cluster.cc:1428] Setting key 4411fe88b5a2dd465c190e1829cc1a23
I20260807 00:02:45.167007 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44921
--builtin_ntp_servers=127.30.204.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260807 00:02:45.169879 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:45.165057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:36433 (local address 127.30.204.126:44921)
0807 00:02:45.165207 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:45.165211 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:45.166664 (+ 1453us) server_negotiation.cc:407] Connection header received
0807 00:02:45.168109 (+ 1445us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:45.168113 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:45.168160 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:45.168243 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:45.169412 (+ 1169us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:45.169417 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:45.169420 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:45.169434 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:45.169457 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:45.169465 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:45.169549 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:45.169683 (+ 134us) server_negotiation.cc:299] Negotiation successful
0807 00:02:45.169733 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:02:45.170814 14588 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:45.165648 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:44921 (local address 127.30.204.65:36433)
0807 00:02:45.166487 (+ 839us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:45.166545 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:45.167885 (+ 1340us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:45.168410 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:45.168418 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:45.168515 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:45.169129 (+ 614us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:45.169131 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:45.169176 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:45.169254 (+ 78us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:45.169536 (+ 282us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:45.169545 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:45.169551 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:02:45.169727 (+ 176us) client_negotiation.cc:240] Negotiation successful
0807 00:02:45.169995 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":140,"threads_started":1}
I20260807 00:02:45.172291 14586 heartbeater.cc:347] Connected to a master server at 127.30.204.126:44921
I20260807 00:02:45.172545 14586 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:45.173015 14586 heartbeater.cc:511] Master 127.30.204.126:44921 requested a full tablet report, sending...
I20260807 00:02:45.174896 14292 ts_manager.cc:194] Registered new tserver with Master: e94bd43d7e5c41bc98c2274526acee73 (127.30.204.65:41391)
W20260807 00:02:45.286824 14590 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:45.287127 14590 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:45.287168 14590 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:45.291337 14590 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:45.291424 14590 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:45.291529 14590 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:45.296633 14590 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14590
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:45.297825 14590 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:45.299275 14590 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:02:45.309621 14590 server_base.cc:1034] running on GCE node
W20260807 00:02:45.309576 14600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:45.309577 14601 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:45.309577 14604 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:45.310400 14590 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:45.311142 14590 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:45.312358 14590 hybrid_clock.cc:648] HybridClock initialized: now 1786060965312334 us; error 35 us; skew 500 ppm
I20260807 00:02:45.314533 14590 webserver.cc:460] Webserver started at http://127.30.204.66:44621/ using document root <none> and password file <none>
I20260807 00:02:45.315165 14590 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:45.315241 14590 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:45.315577 14590 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:45.317906 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "eaddb9d95d1344ba9b03c67432adfa08"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "da02972f64ac02b4d60bb105421f8397"
server_key_iv: "a0a956dc418ceda9529db799e93dd6d8"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.318535 14590 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "eaddb9d95d1344ba9b03c67432adfa08"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "da02972f64ac02b4d60bb105421f8397"
server_key_iv: "a0a956dc418ceda9529db799e93dd6d8"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.323289 14590 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 00:02:45.326610 14611 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.328249 14590 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:45.328405 14590 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "eaddb9d95d1344ba9b03c67432adfa08"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "da02972f64ac02b4d60bb105421f8397"
server_key_iv: "a0a956dc418ceda9529db799e93dd6d8"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.328552 14590 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:45.333587 14590 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:45.334265 14590 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:45.334440 14590 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:45.335070 14590 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:45.336161 14590 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:45.336212 14590 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.336269 14590 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:45.336293 14590 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.345381 14590 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:39665
I20260807 00:02:45.345407 14798 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:39665 every 8 connection(s)
I20260807 00:02:45.346375 14590 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:45.353401 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:45.348578 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:56709 (local address 127.30.204.126:44921)
0807 00:02:45.348760 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:45.348764 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:45.350033 (+ 1269us) server_negotiation.cc:407] Connection header received
0807 00:02:45.351437 (+ 1404us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:45.351440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:45.351519 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:45.351625 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:45.352851 (+ 1226us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:45.352854 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:45.352856 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:45.352872 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:45.352895 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:45.352903 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:45.352990 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:45.353225 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 00:02:45.353279 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 00:02:45.354348 14801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:45.349055 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:44921 (local address 127.30.204.66:56709)
0807 00:02:45.349873 (+ 818us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:45.349919 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:45.351185 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:45.351860 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:45.351868 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:45.351972 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:45.352549 (+ 577us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:45.352551 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:45.352601 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:45.352681 (+ 80us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:45.353016 (+ 335us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:45.353027 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:45.353034 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:02:45.353229 (+ 195us) client_negotiation.cc:240] Negotiation successful
0807 00:02:45.353463 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":327,"thread_start_us":140,"threads_started":1}
I20260807 00:02:45.355191 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 14590
I20260807 00:02:45.355321 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:45.355685 31537 external_mini_cluster.cc:1428] Setting key f028bd054e86289efc219b2f6835a9bd
I20260807 00:02:45.355835 14799 heartbeater.cc:347] Connected to a master server at 127.30.204.126:44921
I20260807 00:02:45.356129 14799 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:45.356594 14799 heartbeater.cc:511] Master 127.30.204.126:44921 requested a full tablet report, sending...
I20260807 00:02:45.357769 14292 ts_manager.cc:194] Registered new tserver with Master: eaddb9d95d1344ba9b03c67432adfa08 (127.30.204.66:39665)
I20260807 00:02:45.357838 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44921
--builtin_ntp_servers=127.30.204.84:35591
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260807 00:02:45.476155 14802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:45.476446 14802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:45.476504 14802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:45.480636 14802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:45.480706 14802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:45.480788 14802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:45.485809 14802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:35591
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:44921
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:45.487016 14802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:45.488523 14802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:45.498797 14811 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:45.498791 14813 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:45.498812 14802 server_base.cc:1034] running on GCE node
W20260807 00:02:45.499011 14809 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:45.499439 14802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:45.500170 14802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:45.501331 14802 hybrid_clock.cc:648] HybridClock initialized: now 1786060965501309 us; error 35 us; skew 500 ppm
I20260807 00:02:45.503293 14802 webserver.cc:460] Webserver started at http://127.30.204.67:36453/ using document root <none> and password file <none>
I20260807 00:02:45.503930 14802 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:45.503997 14802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:45.504194 14802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:45.506332 14802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "df525f297875404785be461bb75fecbc"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "ec30564014589a00185d8a916f5049f3"
server_key_iv: "f304a9e93701d58778ebf82cf57fadc2"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.506904 14802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "df525f297875404785be461bb75fecbc"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "ec30564014589a00185d8a916f5049f3"
server_key_iv: "f304a9e93701d58778ebf82cf57fadc2"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.510893 14802 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:02:45.513775 14820 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.515126 14802 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:45.515249 14802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "df525f297875404785be461bb75fecbc"
format_stamp: "Formatted at 2026-08-07 00:02:45 on dist-test-slave-pww0"
server_key: "ec30564014589a00185d8a916f5049f3"
server_key_iv: "f304a9e93701d58778ebf82cf57fadc2"
server_key_version: "encryptionkey@0"
I20260807 00:02:45.515367 14802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:45.520191 14802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:45.521085 14802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:45.521281 14802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:45.521919 14802 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:45.523061 14802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:45.523118 14802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.523180 14802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:45.523207 14802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:45.532284 14802 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:40431
I20260807 00:02:45.532311 14999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:40431 every 8 connection(s)
I20260807 00:02:45.533453 14802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:45.535750 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 14802
I20260807 00:02:45.535843 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:45.536137 31537 external_mini_cluster.cc:1428] Setting key c61a7c6a3e72b02a3277a0bb457a63d9
I20260807 00:02:45.540056 14343 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:45.535419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:39851 (local address 127.30.204.126:44921)
0807 00:02:45.535602 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:45.535607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:45.536923 (+ 1316us) server_negotiation.cc:407] Connection header received
0807 00:02:45.538521 (+ 1598us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:45.538523 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:45.538562 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:45.538626 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:45.539704 (+ 1078us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:45.539707 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:45.539709 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:02:45.539727 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:45.539748 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:02:45.539757 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:45.539817 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:45.539907 (+ 90us) server_negotiation.cc:299] Negotiation successful
0807 00:02:45.539944 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 00:02:45.541025 15004 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:45.535968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:44921 (local address 127.30.204.67:39851)
0807 00:02:45.536766 (+ 798us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:45.536822 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:45.538299 (+ 1477us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:45.538783 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:45.538791 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:45.538888 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:45.539404 (+ 516us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:02:45.539405 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:45.539448 (+ 43us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:02:45.539564 (+ 116us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:45.539809 (+ 245us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:45.539814 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:45.539818 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:02:45.539946 (+ 128us) client_negotiation.cc:240] Negotiation successful
0807 00:02:45.540180 (+ 234us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":142,"threads_started":1}
I20260807 00:02:45.542168 15001 heartbeater.cc:347] Connected to a master server at 127.30.204.126:44921
I20260807 00:02:45.542433 15001 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:45.542878 15001 heartbeater.cc:511] Master 127.30.204.126:44921 requested a full tablet report, sending...
I20260807 00:02:45.543960 14292 ts_manager.cc:194] Registered new tserver with Master: df525f297875404785be461bb75fecbc (127.30.204.67:40431)
I20260807 00:02:45.550534 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260807 00:02:45.553637 14343 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:02:45.552648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39118 (local address 127.30.204.126:44921)
0807 00:02:45.552773 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:45.552777 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:45.552944 (+ 167us) server_negotiation.cc:407] Connection header received
0807 00:02:45.553043 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:45.553045 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:45.553085 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:45.553161 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:45.553488 (+ 327us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39118: BlockingRecv error: recv got EOF from 127.0.0.1:39118 (error 108)
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:02:45.553889 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 14387
I20260807 00:02:45.589465 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 14590
I20260807 00:02:45.619410 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 14802
I20260807 00:02:45.627146 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 14243
2026-08-07T00:02:45Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (926 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:45 dist-test-slave-pww0 krb5kdc[15017](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:45 dist-test-slave-pww0 krb5kdc[15017](info): set up 2 sockets
Aug 07 00:02:45 dist-test-slave-pww0 krb5kdc[15017](info): commencing operation
krb5kdc: starting...
W20260807 00:02:46.788816 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:46 dist-test-slave-pww0 krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:02:46.927134 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:35665
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:35665
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:47.049808 15038 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:47.050137 15038 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:47.050199 15038 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:47.054554 15038 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:02:47.054643 15038 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:47.054672 15038 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:47.054693 15038 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:02:47.054715 15038 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:02:47.060575 15038 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:35665
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:35665
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15038
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:47.061870 15038 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:47.063097 15038 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:47.073523 15049 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.073523 15045 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.073544 15046 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:47.073752 15038 server_base.cc:1034] running on GCE node
I20260807 00:02:47.074407 15038 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:47.075806 15038 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:47.076994 15038 hybrid_clock.cc:648] HybridClock initialized: now 1786060967076974 us; error 41 us; skew 500 ppm
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:47.080123 15038 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:02:47.081238 15038 webserver.cc:460] Webserver started at http://127.30.204.126:45345/ using document root <none> and password file <none>
I20260807 00:02:47.081867 15038 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:47.081943 15038 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:47.082157 15038 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:47.084685 15038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "c48c890f93184215bfbb529037855d52"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "95afb9cb6ca5f4dbde14d312844e05c0"
server_key_iv: "7c8eda92d04b78926cfe1f30c8abbe78"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.085350 15038 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "c48c890f93184215bfbb529037855d52"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "95afb9cb6ca5f4dbde14d312844e05c0"
server_key_iv: "7c8eda92d04b78926cfe1f30c8abbe78"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.090453 15038 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 00:02:47.093446 15060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.094955 15038 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:02:47.095110 15038 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "c48c890f93184215bfbb529037855d52"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "95afb9cb6ca5f4dbde14d312844e05c0"
server_key_iv: "7c8eda92d04b78926cfe1f30c8abbe78"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.095243 15038 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:47.118870 15038 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:47.122438 15038 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:47.122671 15038 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:47.132484 15038 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:35665
I20260807 00:02:47.132792 15147 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:35665 every 8 connection(s)
I20260807 00:02:47.133893 15038 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:02:47.136272 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 15038
I20260807 00:02:47.136405 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:02:47.136739 31537 external_mini_cluster.cc:1428] Setting key bf8593e1468fdef1f43ef938ae642fea
I20260807 00:02:47.137655 15148 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:47.144078 15148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52: Bootstrap starting.
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060966, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:47.146796 15148 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:47.147634 15148 log.cc:824] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:47.149920 15148 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52: No bootstrap required, opened a new log
I20260807 00:02:47.151752 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:47.138426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46430 (local address 127.30.204.126:35665)
0807 00:02:47.138937 (+ 511us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:47.138954 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:47.138993 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:02:47.139725 (+ 732us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:47.139747 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:47.140105 (+ 358us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:47.140445 (+ 340us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:47.141299 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.142326 (+ 1027us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.143206 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.143758 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.143909 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:47.146634 (+ 2725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:47.146639 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:47.146656 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:47.146690 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:47.148580 (+ 1890us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.149039 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.149045 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.149050 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.149175 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.149458 (+ 283us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.149461 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.149462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.149902 (+ 440us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:47.150084 (+ 182us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:47.150424 (+ 340us) server_negotiation.cc:299] Negotiation successful
0807 00:02:47.150729 (+ 305us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":352,"thread_start_us":166,"threads_started":1}
I20260807 00:02:47.153666 15148 raft_consensus.cc:348] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c48c890f93184215bfbb529037855d52" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35665 } }
I20260807 00:02:47.153950 15148 raft_consensus.cc:374] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:47.154016 15148 raft_consensus.cc:729] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c48c890f93184215bfbb529037855d52, State: Initialized, Role: FOLLOWER
I20260807 00:02:47.154522 15148 consensus_queue.cc:260] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c48c890f93184215bfbb529037855d52" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35665 } }
I20260807 00:02:47.154696 15148 raft_consensus.cc:388] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:02:47.154775 15148 raft_consensus.cc:482] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:02:47.154881 15148 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:47.155997 15148 raft_consensus.cc:504] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c48c890f93184215bfbb529037855d52" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35665 } }
I20260807 00:02:47.156376 15148 leader_election.cc:302] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c48c890f93184215bfbb529037855d52; no voters:
I20260807 00:02:47.156736 15148 leader_election.cc:288] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:02:47.156881 15161 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:47.157159 15161 raft_consensus.cc:686] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [term 1 LEADER]: Becoming Leader. State: Replica: c48c890f93184215bfbb529037855d52, State: Running, Role: LEADER
I20260807 00:02:47.157552 15161 consensus_queue.cc:237] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c48c890f93184215bfbb529037855d52" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35665 } }
I20260807 00:02:47.158103 15148 sys_catalog.cc:564] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:02:47.159408 15162 sys_catalog.cc:455] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c48c890f93184215bfbb529037855d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c48c890f93184215bfbb529037855d52" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35665 } } }
I20260807 00:02:47.159614 15162 sys_catalog.cc:458] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [sys.catalog]: This master's current role is: LEADER
I20260807 00:02:47.160035 15164 sys_catalog.cc:455] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c48c890f93184215bfbb529037855d52. Latest consensus state: current_term: 1 leader_uuid: "c48c890f93184215bfbb529037855d52" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c48c890f93184215bfbb529037855d52" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 35665 } } }
I20260807 00:02:47.160243 15164 sys_catalog.cc:458] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52 [sys.catalog]: This master's current role is: LEADER
W20260807 00:02:47.163522 15186 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:02:47.163625 15186 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:02:47.163722 15174 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:02:47.164705 15174 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:02:47.171579 15174 catalog_manager.cc:1347] Generated new cluster ID: 28b6b62c4fd5458885d83e72245d651f
I20260807 00:02:47.171676 15174 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:02:47.193447 15174 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:02:47.194543 15174 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:02:47.200865 15174 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52: Generated new TSK 0
I20260807 00:02:47.201786 15174 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:02:47.247081 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35665
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:47.391845 15194 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:47.392146 15194 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:47.392190 15194 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:47.396324 15194 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:47.396402 15194 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:47.396466 15194 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:02:47.402133 15194 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15194
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:47.403298 15194 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:47.404444 15194 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:47.415021 15200 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.415021 15201 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.415040 15206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:47.415143 15194 server_base.cc:1034] running on GCE node
I20260807 00:02:47.415823 15194 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:47.416556 15194 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:47.417744 15194 hybrid_clock.cc:648] HybridClock initialized: now 1786060967417722 us; error 39 us; skew 500 ppm
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:47.424301 15194 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:02:47.425477 15194 webserver.cc:460] Webserver started at http://127.30.204.65:45697/ using document root <none> and password file <none>
I20260807 00:02:47.426102 15194 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:47.426182 15194 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:47.426411 15194 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:47.428742 15194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "d67ac070b8054fb6b80a601834f311a0"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "ba33961103b72f5c89b9603e996a69f8"
server_key_iv: "d89545aafbed94994e214b3448a172af"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.429330 15194 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "d67ac070b8054fb6b80a601834f311a0"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "ba33961103b72f5c89b9603e996a69f8"
server_key_iv: "d89545aafbed94994e214b3448a172af"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.433615 15194 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:47.436659 15219 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.438268 15194 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:47.438398 15194 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "d67ac070b8054fb6b80a601834f311a0"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "ba33961103b72f5c89b9603e996a69f8"
server_key_iv: "d89545aafbed94994e214b3448a172af"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.438514 15194 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:47.472080 15194 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:47.475767 15194 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:47.475953 15194 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:47.476565 15194 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:47.477509 15194 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:47.477560 15194 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.477613 15194 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:47.477643 15194 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.488767 15194 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:36011
I20260807 00:02:47.488871 15398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:36011 every 8 connection(s)
I20260807 00:02:47.489845 15194 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:02:47.497380 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 15194
I20260807 00:02:47.497524 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:02:47.497865 31537 external_mini_cluster.cc:1428] Setting key 9019bc3b299d0576a3934a14b34043d2
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:47.506099 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:47.491954 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:36231 (local address 127.30.204.126:35665)
0807 00:02:47.492136 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:47.492141 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:47.493095 (+ 954us) server_negotiation.cc:407] Connection header received
0807 00:02:47.494546 (+ 1451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:47.494550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:47.494604 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:47.494682 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:47.496062 (+ 1380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.496648 (+ 586us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.498550 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.498860 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.498979 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:47.502210 (+ 3231us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:47.502215 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:47.502217 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:47.502253 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:47.503763 (+ 1510us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.504350 (+ 587us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.504354 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.504355 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.504461 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.504842 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.504845 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.504847 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.505093 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:47.505190 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:47.505760 (+ 570us) server_negotiation.cc:299] Negotiation successful
0807 00:02:47.505917 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 00:02:47.507000 15402 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:47.492386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35665 (local address 127.30.204.65:36231)
0807 00:02:47.492903 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:47.492953 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:47.494294 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:47.494858 (+ 564us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:47.494865 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:47.495273 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:47.495867 (+ 594us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:47.495883 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.496828 (+ 945us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.496831 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:47.498374 (+ 1543us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:47.498388 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.499069 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.499073 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:47.499300 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:47.500020 (+ 720us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:47.500045 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:47.501976 (+ 1931us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:47.503908 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:47.503914 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:47.503930 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:47.504237 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:47.504568 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:47.504571 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:47.504572 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:47.504747 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:47.505191 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:47.505197 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:47.505487 (+ 290us) client_negotiation.cc:769] Sending connection context
0807 00:02:47.505705 (+ 218us) client_negotiation.cc:240] Negotiation successful
0807 00:02:47.506040 (+ 335us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":141,"threads_started":1}
I20260807 00:02:47.508394 15399 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35665
I20260807 00:02:47.508665 15399 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:47.509202 15399 heartbeater.cc:511] Master 127.30.204.126:35665 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:02:47.510855 15077 ts_manager.cc:194] Registered new tserver with Master: d67ac070b8054fb6b80a601834f311a0 (127.30.204.65:36011)
I20260807 00:02:47.512156 15077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:36231
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:02:47.535633 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35665
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:47.659304 15406 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:47.659659 15406 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:47.659765 15406 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:47.664155 15406 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:47.664238 15406 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:47.664309 15406 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:02:47.670192 15406 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15406
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:47.671435 15406 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:47.672606 15406 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:47.682845 15413 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.682871 15418 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.682868 15414 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:47.683316 15406 server_base.cc:1034] running on GCE node
I20260807 00:02:47.683815 15406 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:47.684551 15406 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:47.685722 15406 hybrid_clock.cc:648] HybridClock initialized: now 1786060967685700 us; error 36 us; skew 500 ppm
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:47.688862 15406 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:02:47.689952 15406 webserver.cc:460] Webserver started at http://127.30.204.66:42567/ using document root <none> and password file <none>
I20260807 00:02:47.690577 15406 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:47.690649 15406 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:47.690871 15406 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:47.693178 15406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "2758dfd83aef40058dd1109f2d3f8705"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "01d3457dd65ed46b522f757aab59d411"
server_key_iv: "b0cbc3ddc05aee2fcb3511be319f57e8"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.693833 15406 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "2758dfd83aef40058dd1109f2d3f8705"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "01d3457dd65ed46b522f757aab59d411"
server_key_iv: "b0cbc3ddc05aee2fcb3511be319f57e8"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.698163 15406 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:02:47.701092 15429 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.702456 15406 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:02:47.702623 15406 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "2758dfd83aef40058dd1109f2d3f8705"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "01d3457dd65ed46b522f757aab59d411"
server_key_iv: "b0cbc3ddc05aee2fcb3511be319f57e8"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.702752 15406 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:47.726593 15406 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:47.730329 15406 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:47.730544 15406 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:47.731215 15406 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:47.732221 15406 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:47.732280 15406 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.732327 15406 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:47.732352 15406 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.742201 15406 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:33421
I20260807 00:02:47.742241 15621 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:33421 every 8 connection(s)
I20260807 00:02:47.743216 15406 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:02:47.744375 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 15406
I20260807 00:02:47.744485 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:02:47.744776 31537 external_mini_cluster.cc:1428] Setting key 2bf96f57fc74fe4178055f508173fe3b
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:02:47.761063 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:47.745257 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:48377 (local address 127.30.204.126:35665)
0807 00:02:47.745419 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:47.745423 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:47.746513 (+ 1090us) server_negotiation.cc:407] Connection header received
0807 00:02:47.748039 (+ 1526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:47.748042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:47.748106 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:47.748203 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:47.750088 (+ 1885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.750678 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.752580 (+ 1902us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.752855 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.752960 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:47.756690 (+ 3730us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:47.756694 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:47.756696 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:47.756729 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:47.758259 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.759013 (+ 754us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.759018 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.759020 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.759130 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.759696 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.759699 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.759701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.759917 (+ 216us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:47.760001 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:47.760769 (+ 768us) server_negotiation.cc:299] Negotiation successful
0807 00:02:47.760926 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 00:02:47.762214 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:47.745788 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35665 (local address 127.30.204.66:48377)
0807 00:02:47.746341 (+ 553us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:47.746401 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:47.747804 (+ 1403us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:47.748468 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:47.748478 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:47.749093 (+ 615us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:47.749871 (+ 778us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:47.749895 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.750842 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.750846 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:47.752405 (+ 1559us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:47.752422 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.753060 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.753066 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:47.753426 (+ 360us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:47.754214 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:47.754240 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:47.756377 (+ 2137us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:47.758422 (+ 2045us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:47.758431 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:47.758454 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:47.758861 (+ 407us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:47.759266 (+ 405us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:47.759269 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:47.759271 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:47.759549 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:47.760050 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:47.760056 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:47.760408 (+ 352us) client_negotiation.cc:769] Sending connection context
0807 00:02:47.760664 (+ 256us) client_negotiation.cc:240] Negotiation successful
0807 00:02:47.761044 (+ 380us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":141,"threads_started":1}
I20260807 00:02:47.763548 15622 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35665
I20260807 00:02:47.763860 15622 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:47.764508 15622 heartbeater.cc:511] Master 127.30.204.126:35665 requested a full tablet report, sending...
I20260807 00:02:47.765827 15077 ts_manager.cc:194] Registered new tserver with Master: 2758dfd83aef40058dd1109f2d3f8705 (127.30.204.66:33421)
I20260807 00:02:47.766644 15077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:48377
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:02:47.779737 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35665
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41707
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:02:47.900732 15634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:02:47.901057 15634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:02:47.901125 15634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:02:47.905568 15634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:02:47.905670 15634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:02:47.905772 15634 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:02:47.911619 15634 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41707
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:35665
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:02:47.912868 15634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:02:47.914019 15634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:02:47.924307 15643 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.924305 15642 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:02:47.924305 15645 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:02:47.924784 15634 server_base.cc:1034] running on GCE node
I20260807 00:02:47.925252 15634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:02:47.925978 15634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:02:47.927165 15634 hybrid_clock.cc:648] HybridClock initialized: now 1786060967927130 us; error 50 us; skew 500 ppm
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:02:47.930403 15634 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:02:47.931576 15634 webserver.cc:460] Webserver started at http://127.30.204.67:33215/ using document root <none> and password file <none>
I20260807 00:02:47.932272 15634 fs_manager.cc:359] Metadata directory not provided
I20260807 00:02:47.932351 15634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:02:47.932570 15634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:02:47.934851 15634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "a3b04400248a4cf798d9fe6c7f1b69c7"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "ce727cde01daef25e9ec5f2eba092418"
server_key_iv: "fafc68075b29f39a4fe9ce6cceb97f16"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.935527 15634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "a3b04400248a4cf798d9fe6c7f1b69c7"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "ce727cde01daef25e9ec5f2eba092418"
server_key_iv: "fafc68075b29f39a4fe9ce6cceb97f16"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.939735 15634 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:02:47.942580 15654 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.944025 15634 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:02:47.944166 15634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "a3b04400248a4cf798d9fe6c7f1b69c7"
format_stamp: "Formatted at 2026-08-07 00:02:47 on dist-test-slave-pww0"
server_key: "ce727cde01daef25e9ec5f2eba092418"
server_key_iv: "fafc68075b29f39a4fe9ce6cceb97f16"
server_key_version: "encryptionkey@0"
I20260807 00:02:47.944281 15634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:02:47.961357 15634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:02:47.965214 15634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:02:47.965405 15634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:02:47.966035 15634 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:02:47.966954 15634 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:02:47.967017 15634 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.967063 15634 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:02:47.967077 15634 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:02:47.977417 15634 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:34205
I20260807 00:02:47.977571 15833 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:34205 every 8 connection(s)
I20260807 00:02:47.978650 15634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:02:47.988405 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 15634
I20260807 00:02:47.988525 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:02:47.988875 31537 external_mini_cluster.cc:1428] Setting key e45856f42bf0c50fc3c6750490230e32
Aug 07 00:02:47 dist-test-slave-pww0 krb5kdc[15017](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060967, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:02:47.994894 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:47.980913 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:56059 (local address 127.30.204.126:35665)
0807 00:02:47.981169 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:47.981183 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:47.982155 (+ 972us) server_negotiation.cc:407] Connection header received
0807 00:02:47.983728 (+ 1573us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:47.983732 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:47.983806 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:47.983885 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:47.985273 (+ 1388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.985823 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.987105 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.987342 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.987439 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:47.990633 (+ 3194us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:47.990637 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:47.990640 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:47.990676 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:47.992277 (+ 1601us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.992971 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.992974 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.992975 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.993074 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:47.993590 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:47.993593 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:47.993594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:47.993845 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:47.993937 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:47.994606 (+ 669us) server_negotiation.cc:299] Negotiation successful
0807 00:02:47.994760 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102}
I20260807 00:02:47.995891 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:47.981355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:35665 (local address 127.30.204.67:56059)
0807 00:02:47.981937 (+ 582us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:47.981991 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:47.983395 (+ 1404us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:47.984098 (+ 703us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:47.984107 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:47.984543 (+ 436us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:02:47.985097 (+ 554us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:47.985113 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.985970 (+ 857us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.985973 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:47.986958 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:47.986970 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:47.987550 (+ 580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:47.987556 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:47.987751 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:47.988498 (+ 747us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:02:47.988525 (+ 27us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:02:47.990450 (+ 1925us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:02:47.992488 (+ 2038us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:47.992495 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:47.992513 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:47.992841 (+ 328us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:47.993217 (+ 376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:02:47.993220 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:02:47.993221 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:02:47.993435 (+ 214us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:02:47.993979 (+ 544us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:02:47.993986 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:02:47.994317 (+ 331us) client_negotiation.cc:769] Sending connection context
0807 00:02:47.994514 (+ 197us) client_negotiation.cc:240] Negotiation successful
0807 00:02:47.994864 (+ 350us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":137,"threads_started":1}
I20260807 00:02:47.997174 15835 heartbeater.cc:347] Connected to a master server at 127.30.204.126:35665
I20260807 00:02:47.997421 15835 heartbeater.cc:464] Registering TS with master...
I20260807 00:02:47.997951 15835 heartbeater.cc:511] Master 127.30.204.126:35665 requested a full tablet report, sending...
I20260807 00:02:47.999076 15077 ts_manager.cc:194] Registered new tserver with Master: a3b04400248a4cf798d9fe6c7f1b69c7 (127.30.204.67:34205)
I20260807 00:02:47.999764 15077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:56059
I20260807 00:02:48.003820 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:02:48.013964 15157 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.005882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46436 (local address 127.30.204.126:35665)
0807 00:02:48.006014 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:48.006019 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:48.006255 (+ 236us) server_negotiation.cc:407] Connection header received
0807 00:02:48.006393 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:48.006396 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:48.006443 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:48.006512 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:02:48.007273 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.007884 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.008776 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.009005 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.009101 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.010561 (+ 1460us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:02:48.010564 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:02:48.010567 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:02:48.010598 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:02:48.012194 (+ 1596us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:48.012645 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:48.012649 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:48.012651 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:48.012746 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:02:48.013011 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:02:48.013014 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:02:48.013016 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:02:48.013287 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:02:48.013415 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:48.013662 (+ 247us) server_negotiation.cc:299] Negotiation successful
0807 00:02:48.013797 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:02:48.017259 15077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46436:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:02:48.020488 15077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:02:48.037349 15855 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.030894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:34205 (local address 127.0.0.1:47280)
0807 00:02:48.031622 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:48.031685 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:48.031850 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:48.033082 (+ 1232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:48.033085 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:48.033106 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:48.033270 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.033279 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.034767 (+ 1488us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.034770 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.035888 (+ 1118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.035898 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.036904 (+ 1006us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.036908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.036991 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.037012 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 00:02:48.037082 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 00:02:48.037142 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":614,"mutex_wait_us":53,"spinlock_wait_cycles":1152,"thread_start_us":89,"threads_started":1}
I20260807 00:02:48.037402 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.030625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59288 (local address 127.30.204.66:33421)
0807 00:02:48.031092 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:48.031105 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:48.032140 (+ 1035us) server_negotiation.cc:407] Connection header received
0807 00:02:48.032230 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:48.032235 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:48.032392 (+ 157us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:48.032530 (+ 138us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:48.033413 (+ 883us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.034613 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.035865 (+ 1252us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.036500 (+ 635us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.036612 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.036771 (+ 159us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:48.037113 (+ 342us) server_negotiation.cc:299] Negotiation successful
0807 00:02:48.037233 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":130,"threads_started":1}
I20260807 00:02:48.037531 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.030211 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59686 (local address 127.30.204.65:36011)
0807 00:02:48.031268 (+ 1057us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:48.031275 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:48.031796 (+ 521us) server_negotiation.cc:407] Connection header received
0807 00:02:48.032054 (+ 258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:48.032060 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:48.032346 (+ 286us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:48.032530 (+ 184us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:48.032897 (+ 367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.033951 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.035640 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.036444 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.036565 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.036739 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:48.037292 (+ 553us) server_negotiation.cc:299] Negotiation successful
0807 00:02:48.037397 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":953,"thread_start_us":174,"threads_started":1}
I20260807 00:02:48.037847 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.030449 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:33421 (local address 127.0.0.1:59288)
0807 00:02:48.032031 (+ 1582us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:48.032042 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:48.032110 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:48.032535 (+ 425us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:48.032543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:48.032580 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:48.033282 (+ 702us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.033289 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.034793 (+ 1504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.034796 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.035725 (+ 929us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.035736 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.036838 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.036842 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.036930 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.036953 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:02:48.037022 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:02:48.037103 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":881,"mutex_wait_us":627,"spinlock_wait_cycles":1446400,"thread_start_us":124,"threads_started":1}
I20260807 00:02:48.037807 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.030056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36011 (local address 127.0.0.1:59686)
0807 00:02:48.031622 (+ 1566us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:48.031685 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:48.031850 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:48.032535 (+ 685us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:48.032543 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:48.032580 (+ 37us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:48.032775 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.032783 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.034073 (+ 1290us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.034076 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.035504 (+ 1428us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.035517 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.036585 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.036589 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.036720 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.036764 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 00:02:48.036883 (+ 119us) client_negotiation.cc:240] Negotiation successful
0807 00:02:48.037670 (+ 787us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":733,"mutex_wait_us":4,"spinlock_wait_cycles":13440,"thread_start_us":111,"threads_started":1}
I20260807 00:02:48.038371 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.031092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47280 (local address 127.30.204.67:34205)
0807 00:02:48.031498 (+ 406us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:48.031510 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:48.031842 (+ 332us) server_negotiation.cc:407] Connection header received
0807 00:02:48.032729 (+ 887us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:48.032735 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:48.032945 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:48.033066 (+ 121us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:48.033424 (+ 358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.034613 (+ 1189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.036010 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.036787 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.036888 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.037020 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:48.037919 (+ 899us) server_negotiation.cc:299] Negotiation successful
0807 00:02:48.038185 (+ 266us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":177,"threads_started":1}
I20260807 00:02:48.041421 15508 tablet_service.cc:1467] Processing CreateTablet for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1 (DEFAULT_TABLE table=test-table [id=43799b15f8ff4b339992869d1f706f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:48.041422 15294 tablet_service.cc:1467] Processing CreateTablet for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1 (DEFAULT_TABLE table=test-table [id=43799b15f8ff4b339992869d1f706f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:48.042716 15294 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:48.042716 15508 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:48.043643 15733 tablet_service.cc:1467] Processing CreateTablet for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1 (DEFAULT_TABLE table=test-table [id=43799b15f8ff4b339992869d1f706f3b]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:02:48.044637 15733 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:02:48.050547 15858 tablet_bootstrap.cc:492] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0: Bootstrap starting.
I20260807 00:02:48.051734 15859 tablet_bootstrap.cc:492] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Bootstrap starting.
I20260807 00:02:48.051985 15860 tablet_bootstrap.cc:492] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Bootstrap starting.
I20260807 00:02:48.053242 15858 tablet_bootstrap.cc:654] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:48.054164 15860 tablet_bootstrap.cc:654] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:48.054163 15858 log.cc:824] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:48.054210 15859 tablet_bootstrap.cc:654] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Neither blocks nor log segments found. Creating new log.
I20260807 00:02:48.054987 15859 log.cc:824] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:48.055341 15860 log.cc:824] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:02:48.056592 15858 tablet_bootstrap.cc:492] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0: No bootstrap required, opened a new log
I20260807 00:02:48.056797 15858 ts_tablet_manager.cc:1397] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0: Time spent bootstrapping tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:02:48.056991 15859 tablet_bootstrap.cc:492] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: No bootstrap required, opened a new log
I20260807 00:02:48.057190 15859 ts_tablet_manager.cc:1397] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:02:48.057412 15860 tablet_bootstrap.cc:492] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: No bootstrap required, opened a new log
I20260807 00:02:48.057617 15860 ts_tablet_manager.cc:1397] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 00:02:48.060959 15858 raft_consensus.cc:348] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.061198 15858 raft_consensus.cc:374] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:48.061251 15858 raft_consensus.cc:729] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d67ac070b8054fb6b80a601834f311a0, State: Initialized, Role: FOLLOWER
I20260807 00:02:48.061390 15860 raft_consensus.cc:348] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.061390 15859 raft_consensus.cc:348] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.061659 15860 raft_consensus.cc:374] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:48.061662 15859 raft_consensus.cc:374] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:02:48.061712 15860 raft_consensus.cc:729] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a3b04400248a4cf798d9fe6c7f1b69c7, State: Initialized, Role: FOLLOWER
I20260807 00:02:48.061722 15859 raft_consensus.cc:729] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2758dfd83aef40058dd1109f2d3f8705, State: Initialized, Role: FOLLOWER
I20260807 00:02:48.061659 15858 consensus_queue.cc:260] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.062165 15860 consensus_queue.cc:260] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.062165 15859 consensus_queue.cc:260] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.062636 15858 ts_tablet_manager.cc:1428] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 00:02:48.062930 15835 heartbeater.cc:503] Master 127.30.204.126:35665 was elected leader, sending a full tablet report...
I20260807 00:02:48.062930 15399 heartbeater.cc:503] Master 127.30.204.126:35665 was elected leader, sending a full tablet report...
I20260807 00:02:48.063205 15859 ts_tablet_manager.cc:1428] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:02:48.063210 15860 ts_tablet_manager.cc:1428] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Time spent starting tablet: real 0.006s user 0.000s sys 0.005s
I20260807 00:02:48.063633 15622 heartbeater.cc:503] Master 127.30.204.126:35665 was elected leader, sending a full tablet report...
I20260807 00:02:48.065555 15871 raft_consensus.cc:482] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:02:48.065723 15871 raft_consensus.cc:504] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.067106 15871 leader_election.cc:288] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d67ac070b8054fb6b80a601834f311a0 (127.30.204.65:36011), a3b04400248a4cf798d9fe6c7f1b69c7 (127.30.204.67:34205)
I20260807 00:02:48.071900 15626 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.067349 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36011 (local address 127.30.204.66:47097)
0807 00:02:48.067638 (+ 289us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:48.067655 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:48.067746 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:48.068210 (+ 464us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:48.068213 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:48.068241 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:48.068361 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.068368 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.069318 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.069322 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.070500 (+ 1178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.070509 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.071489 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.071493 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.071599 (+ 106us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.071621 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:02:48.071683 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:02:48.071759 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203}
I20260807 00:02:48.072353 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.067492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:47097 (local address 127.30.204.65:36011)
0807 00:02:48.067908 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:48.067913 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:48.067927 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:02:48.067975 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:48.067978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:48.068023 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:48.068089 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:48.068488 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.069180 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.070635 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.071217 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.071350 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.071408 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:48.072147 (+ 739us) server_negotiation.cc:299] Negotiation successful
0807 00:02:48.072238 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331}
I20260807 00:02:48.072896 15330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08585bb68c94ae5b0ee12cfb8f3d2f1" candidate_uuid: "2758dfd83aef40058dd1109f2d3f8705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67ac070b8054fb6b80a601834f311a0" is_pre_election: true
I20260807 00:02:48.073197 15330 raft_consensus.cc:2454] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2758dfd83aef40058dd1109f2d3f8705 in term 0.
I20260807 00:02:48.073679 15431 leader_election.cc:302] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2758dfd83aef40058dd1109f2d3f8705, d67ac070b8054fb6b80a601834f311a0; no voters:
I20260807 00:02:48.073963 15871 raft_consensus.cc:2781] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:02:48.074069 15871 raft_consensus.cc:482] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:02:48.074121 15871 raft_consensus.cc:3037] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:48.074749 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.067665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:34205 (local address 127.30.204.66:51297)
0807 00:02:48.069737 (+ 2072us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:48.069752 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:02:48.069887 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:02:48.071436 (+ 1549us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:02:48.071439 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:02:48.071480 (+ 41us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:02:48.071588 (+ 108us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.071594 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.072721 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.072724 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.073570 (+ 846us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:02:48.073578 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.074380 (+ 802us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.074383 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:02:48.074466 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.074490 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:02:48.074553 (+ 63us) client_negotiation.cc:240] Negotiation successful
0807 00:02:48.074610 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1978,"thread_start_us":128,"threads_started":1}
I20260807 00:02:48.074783 15857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.068102 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:51297 (local address 127.30.204.67:34205)
0807 00:02:48.068335 (+ 233us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:48.068340 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:48.069824 (+ 1484us) server_negotiation.cc:407] Connection header received
0807 00:02:48.071085 (+ 1261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:48.071088 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:48.071137 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:48.071182 (+ 45us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:02:48.071714 (+ 532us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.072409 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.073692 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.074253 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.074375 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.074427 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:48.074580 (+ 153us) server_negotiation.cc:299] Negotiation successful
0807 00:02:48.074656 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260807 00:02:48.075343 15762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08585bb68c94ae5b0ee12cfb8f3d2f1" candidate_uuid: "2758dfd83aef40058dd1109f2d3f8705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" is_pre_election: true
I20260807 00:02:48.075543 15871 raft_consensus.cc:504] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.075685 15762 raft_consensus.cc:2454] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2758dfd83aef40058dd1109f2d3f8705 in term 0.
I20260807 00:02:48.075917 15871 leader_election.cc:288] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [CANDIDATE]: Term 1 election: Requested vote from peers d67ac070b8054fb6b80a601834f311a0 (127.30.204.65:36011), a3b04400248a4cf798d9fe6c7f1b69c7 (127.30.204.67:34205)
I20260807 00:02:48.076294 15762 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08585bb68c94ae5b0ee12cfb8f3d2f1" candidate_uuid: "2758dfd83aef40058dd1109f2d3f8705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7"
I20260807 00:02:48.076432 15762 raft_consensus.cc:3037] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:48.076642 15330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a08585bb68c94ae5b0ee12cfb8f3d2f1" candidate_uuid: "2758dfd83aef40058dd1109f2d3f8705" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d67ac070b8054fb6b80a601834f311a0"
I20260807 00:02:48.076794 15330 raft_consensus.cc:3037] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:02:48.077570 15762 raft_consensus.cc:2454] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2758dfd83aef40058dd1109f2d3f8705 in term 1.
I20260807 00:02:48.077831 15431 leader_election.cc:302] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2758dfd83aef40058dd1109f2d3f8705, a3b04400248a4cf798d9fe6c7f1b69c7; no voters:
I20260807 00:02:48.077852 15330 raft_consensus.cc:2454] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2758dfd83aef40058dd1109f2d3f8705 in term 1.
I20260807 00:02:48.077998 15871 raft_consensus.cc:2781] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:02:48.078254 15871 raft_consensus.cc:686] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 1 LEADER]: Becoming Leader. State: Replica: 2758dfd83aef40058dd1109f2d3f8705, State: Running, Role: LEADER
I20260807 00:02:48.078583 15871 consensus_queue.cc:237] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } }
I20260807 00:02:48.081894 15088 catalog_manager.cc:5495] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2758dfd83aef40058dd1109f2d3f8705 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "2758dfd83aef40058dd1109f2d3f8705" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2758dfd83aef40058dd1109f2d3f8705" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33421 } health_report { overall_health: HEALTHY } } }
I20260807 00:02:48.102802 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.099094 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59302 (local address 127.30.204.66:33421)
0807 00:02:48.099229 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 00:02:48.099234 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:02:48.099250 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:02:48.099438 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:02:48.099440 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:02:48.099536 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:02:48.099658 (+ 122us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:02:48.099948 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.100462 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.101326 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:02:48.101542 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:02:48.101635 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:02:48.101957 (+ 322us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:02:48.102395 (+ 438us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:02:48.102506 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 00:02:48.102579 (+ 73us) server_negotiation.cc:299] Negotiation successful
0807 00:02:48.102654 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:02:48.109479 15330 raft_consensus.cc:1261] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P d67ac070b8054fb6b80a601834f311a0 [term 1 FOLLOWER]: Refusing update from remote peer 2758dfd83aef40058dd1109f2d3f8705: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:48.109479 15762 raft_consensus.cc:1261] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 1 FOLLOWER]: Refusing update from remote peer 2758dfd83aef40058dd1109f2d3f8705: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:02:48.110275 15871 consensus_queue.cc:1035] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d67ac070b8054fb6b80a601834f311a0" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36011 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:48.110507 15873 consensus_queue.cc:1035] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a3b04400248a4cf798d9fe6c7f1b69c7" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34205 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:02:48.118645 15879 mvcc.cc:205] Tried to move back new op lower bound from 7315705725368348672 to 7315705725252513792. Current Snapshot: MvccSnapshot[applied={T|T < 7315705725368348672}]
I20260807 00:02:48.118925 15880 mvcc.cc:205] Tried to move back new op lower bound from 7315705725368348672 to 7315705725252513792. Current Snapshot: MvccSnapshot[applied={T|T < 7315705725368348672}]
I20260807 00:02:48.121318 15881 mvcc.cc:205] Tried to move back new op lower bound from 7315705725368348672 to 7315705725252513792. Current Snapshot: MvccSnapshot[applied={T|T < 7315705725368348672}]
I20260807 00:02:48.128770 15077 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:48.128968 15077 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46436:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:02:48.131841 15077 catalog_manager.cc:5782] T 00000000000000000000000000000000 P c48c890f93184215bfbb529037855d52: Sending DeleteTablet for 3 replicas of tablet a08585bb68c94ae5b0ee12cfb8f3d2f1
I20260807 00:02:48.133235 15733 tablet_service.cc:1514] Processing DeleteTablet for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:48 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:47280
I20260807 00:02:48.133230 15508 tablet_service.cc:1514] Processing DeleteTablet for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:02:48 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:59288
I20260807 00:02:48.133625 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 15194
I20260807 00:02:48.134294 15896 tablet_replica.cc:330] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: stopping tablet replica
I20260807 00:02:48.134714 15896 raft_consensus.cc:2229] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:02:48.135252 15896 raft_consensus.cc:2258] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:02:48.142352 15896 ts_tablet_manager.cc:1905] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:48.145692 15896 ts_tablet_manager.cc:1918] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:48.145942 15896 log.cc:1196] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/a08585bb68c94ae5b0ee12cfb8f3d2f1
I20260807 00:02:48.146391 15896 ts_tablet_manager.cc:1939] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P 2758dfd83aef40058dd1109f2d3f8705: Deleting consensus metadata
I20260807 00:02:48.148025 15064 catalog_manager.cc:4841] TS 2758dfd83aef40058dd1109f2d3f8705 (127.30.204.66:33421): tablet a08585bb68c94ae5b0ee12cfb8f3d2f1 (table test-table [id=43799b15f8ff4b339992869d1f706f3b]) successfully deleted
I20260807 00:02:48.148729 15898 tablet_replica.cc:330] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: stopping tablet replica
I20260807 00:02:48.149188 15898 raft_consensus.cc:2229] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:02:48.149633 15898 raft_consensus.cc:2258] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260807 00:02:48.150830 15062 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:36011 (error 108)
I20260807 00:02:48.151365 15898 ts_tablet_manager.cc:1905] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:02:48.151814 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 15406
I20260807 00:02:48.151984 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:02:48.151697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36011 (local address 127.0.0.1:59700)
0807 00:02:48.151821 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 00:02:48.151899 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:36011: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260807 00:02:48.152141 15062 catalog_manager.cc:4584] TS d67ac070b8054fb6b80a601834f311a0 (127.30.204.65:36011): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a08585bb68c94ae5b0ee12cfb8f3d2f1: Network error: Client connection negotiation failed: client connection to 127.30.204.65:36011: connect: Connection refused (error 111)
I20260807 00:02:48.173604 15898 ts_tablet_manager.cc:1918] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:02:48.173830 15898 log.cc:1196] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/a08585bb68c94ae5b0ee12cfb8f3d2f1
I20260807 00:02:48.174230 15898 ts_tablet_manager.cc:1939] T a08585bb68c94ae5b0ee12cfb8f3d2f1 P a3b04400248a4cf798d9fe6c7f1b69c7: Deleting consensus metadata
I20260807 00:02:48.179627 15062 catalog_manager.cc:4841] TS a3b04400248a4cf798d9fe6c7f1b69c7 (127.30.204.67:34205): tablet a08585bb68c94ae5b0ee12cfb8f3d2f1 (table test-table [id=43799b15f8ff4b339992869d1f706f3b]) successfully deleted
I20260807 00:02:48.184036 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 15634
I20260807 00:02:48.199990 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 15038
2026-08-07T00:02:48Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (2572 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:02:48 dist-test-slave-pww0 krb5kdc[15909](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:02:48 dist-test-slave-pww0 krb5kdc[15909](info): set up 2 sockets
Aug 07 00:02:48 dist-test-slave-pww0 krb5kdc[15909](info): commencing operation
krb5kdc: starting...
W20260807 00:02:49.361948 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:02:49 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786060969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:02:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:02:49Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/rangeradmin_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/rangeradmin_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/rangerlookup_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/rangerlookup_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
I20260807 00:02:49.550349 31537 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskl_Zz3S/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/postgres -l logfile start
2026-08-07 00:02:52.197 UTC [15954] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 00:02:52.197 UTC [15954] LOG: listening on IPv4 address "127.30.204.84", port 46775
2026-08-07 00:02:52.201 UTC [15954] LOG: listening on Unix socket "/tmp/.s.PGSQL.46775"
2026-08-07 00:02:52.223 UTC [15957] LOG: database system was shut down at 2026-08-07 00:02:50 UTC
2026-08-07 00:02:52.227 UTC [15954] LOG: database system is ready to accept connections
I20260807 00:02:53.290192 31537 mini_postgres.cc:96] Postgres bound to 46775
2026-08-07 00:02:53.298 UTC [15970] FATAL: database "slave" does not exist
127.30.204.84:46775 - accepting connections
I20260807 00:02:53.299721 31537 mini_ranger.cc:161] Starting Ranger...
I20260807 00:02:53.317662 31537 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 00:02:53.808524 31537 mini_ranger.cc:87] Created ranger Postgres database
I20260807 00:02:53.808615 31537 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-admin
2026-08-07 00:02:54,154 [I] DB FLAVOR :POSTGRES
2026-08-07 00:02:54,155 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:02:54,155 [I] Checking connection..
2026-08-07 00:02:54,155 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 00:02:54,439 [I] Checking connection passed.
2026-08-07 00:02:54,439 [I] --------- Verifying version history table ---------
2026-08-07 00:02:54,439 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:02:54,689 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:02:54,689 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:02:54,976 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:02:54,977 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 00:02:54,977 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 00:02:55.235 UTC [16085] WARNING: there is no transaction in progress
2026-08-07 00:02:55,245 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 00:02:55,245 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:02:55,519 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 00:02:55,520 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:02:55,520 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 00:02:55,795 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 00:02:56,066 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pww0.c.gcp-upstream.internal','N') ;"
2026-08-07 00:02:56,323 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:02:56,589 [I] Table x_portal_user does not exist in database ranger
2026-08-07 00:02:56,590 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:02:56,851 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 00:02:56,852 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:02:57,132 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:02:57,408 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 00:02:57,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 00:02:58.464 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.482 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.499 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.516 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.534 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.552 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.612 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.619 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.635 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.647 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.659 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.674 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.684 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.691 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.699 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:58.706 UTC [16254] WARNING: there is no transaction in progress
2026-08-07 00:02:59,428 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 00:02:59,429 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 00:02:59,715 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 00:02:59,715 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:02:59,998 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 00:02:59,998 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:03:00,269 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 00:03:00,269 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:03:00,522 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:03:00,788 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pww0.c.gcp-upstream.internal';"
2026-08-07 00:03:01,035 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 00:03:01,035 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 00:03:01,293 [I] DB_PATCHES have already been applied
I20260807 00:03:01.299885 31537 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-admin:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskl_Zz3S/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 00:03:01.299962 31537 mini_ranger.cc:193] Using host: 127.30.204.84
I20260807 00:03:01.304816 31537 mini_ranger.cc:227] Ranger admin URL: http://127.30.204.84:33883
Aug 07, 2026 12:03:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:03:01 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:03:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 12:03:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 12:03:01 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 12:03:02 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 12:03:02 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 00:03:02.584357 31537 mini_ranger.cc:160] Time spent starting Ranger: real 9.285s user 0.000s sys 0.014s
Aug 07, 2026 12:03:24 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 12:03:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Aug 07, 2026 12:03:24 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 12:03:24 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 12:03:25 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 00:03:25.915551 31537 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/rangerkms_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/rangerkms_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.204.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.204.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 00:03:26 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061006, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 00:03:26.011919 31537 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 00:03:26.023991 31537 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 00:03:26.915752 31537 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 00:03:26.915848 31537 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms
I20260807 00:03:26.915966 31537 mini_ranger_kms.cc:227] Using postgres at 127.30.204.84:46775
2026-08-07 00:03:27,407 [I] DB FLAVOR :POSTGRES
2026-08-07 00:03:27,407 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:03:27,407 [I] Checking connection
2026-08-07 00:03:27,408 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 00:03:27,662 [I] connection success
2026-08-07 00:03:27,663 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 00:03:27,663 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 00:03:27,663 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 00:03:27,935 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 00:03:27,935 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:03:27,935 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 00:03:27,935 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:46775/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 00:03:28,238 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 00:03:28.751195 31537 mini_ranger_kms.cc:310] Created kms service
I20260807 00:03:29.010376 31537 mini_ranger_kms.cc:326] Created kudu user
I20260807 00:03:29.060364 31537 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 00:03:29 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061006, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:03:29.511502 31537 mini_ranger_kms.cc:384] Added ranger policy
I20260807 00:03:29.511826 31537 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms:/tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 00:03:29.511885 31537 mini_ranger_kms.cc:240] Using host: 127.30.204.84
I20260807 00:03:29.516607 31537 mini_ranger_kms.cc:276] Ranger KMS PID: 16564
I20260807 00:03:29.516769 31537 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.204.84:47131
Aug 07, 2026 12:03:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:03:30 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:03:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 12:03:30 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 12:03:31 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 12:03:31 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 00:03:31 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786061011, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.204.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:03:32.109189 31537 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 6.097s user 0.026s sys 0.236s
I20260807 00:03:32.109344 31537 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 00:03:32.109477 31537 mini_ranger_kms.cc:401] 127.30.204.84:47131/kms/v1/keys
Aug 07 00:03:32 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061011, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.204.84@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
Aug 07 00:03:32 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061011, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.204.84@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
Aug 07 00:03:35 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:03:35.282356 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:42643
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:42643
--ranger_config_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:47131/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:03:35.405285 16646 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:03:35.405660 16646 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:03:35.405706 16646 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:03:35.409835 16646 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:03:35.409924 16646 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:03:35.409945 16646 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:03:35.409963 16646 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:03:35.409981 16646 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:03:35.415838 16646 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.204.84:47131/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:42643
--ranger_config_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:42643
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16646
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:03:35.417007 16646 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:03:35.418077 16646 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:03:35.427811 16646 server_base.cc:1034] running on GCE node
W20260807 00:03:35.427882 16652 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:03:35.427855 16651 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:03:35.427855 16656 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:03:35.428710 16646 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:03:35.429916 16646 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:03:35.431079 16646 hybrid_clock.cc:648] HybridClock initialized: now 1786061015431042 us; error 52 us; skew 500 ppm
Aug 07 00:03:35 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:03:35.433924 16646 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:03:35.434893 16646 webserver.cc:460] Webserver started at http://127.30.204.126:33719/ using document root <none> and password file <none>
I20260807 00:03:35.435431 16646 fs_manager.cc:359] Metadata directory not provided
I20260807 00:03:35.435524 16646 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:03:35.435844 16646 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:03:35 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061015, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:03:35.494310 16646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "ffd7bd1a42094fb7aa13dee38b7d1111"
format_stamp: "Formatted at 2026-08-07 00:03:35 on dist-test-slave-pww0"
server_key: "eb82a1c02acc8073d5be17d7a7bb287f"
server_key_iv: "9966642e71197267c742f53a193f67e9"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:35.495038 16646 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "ffd7bd1a42094fb7aa13dee38b7d1111"
format_stamp: "Formatted at 2026-08-07 00:03:35 on dist-test-slave-pww0"
server_key: "eb82a1c02acc8073d5be17d7a7bb287f"
server_key_iv: "9966642e71197267c742f53a193f67e9"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:35.499282 16646 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:03:35.516218 16674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:35.517668 16646 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:03:35.517812 16646 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "ffd7bd1a42094fb7aa13dee38b7d1111"
format_stamp: "Formatted at 2026-08-07 00:03:35 on dist-test-slave-pww0"
server_key: "eb82a1c02acc8073d5be17d7a7bb287f"
server_key_iv: "9966642e71197267c742f53a193f67e9"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:35.517949 16646 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:03:35.528069 16646 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:03:35.531091 16646 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:03:35.531313 16646 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:03:35.538635 16646 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:42643
I20260807 00:03:35.538687 16762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:42643 every 8 connection(s)
I20260807 00:03:35.539752 16646 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:03:35.541700 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 16646
I20260807 00:03:35.541819 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:03:35.542094 31537 external_mini_cluster.cc:1428] Setting key c1a88bea00e6aa59ff943dfd8d910255
I20260807 00:03:35.542860 16764 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:03:35.550392 16764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111: Bootstrap starting.
Aug 07 00:03:35 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061015, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:03:35.553287 16764 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111: Neither blocks nor log segments found. Creating new log.
I20260807 00:03:35.554194 16764 log.cc:824] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111: Log is configured to *not* fsync() on all Append() calls
I20260807 00:03:35.556348 16764 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111: No bootstrap required, opened a new log
I20260807 00:03:35.557772 16769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:35.543615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42532 (local address 127.30.204.126:42643)
0807 00:03:35.544072 (+ 457us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:35.544093 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:35.544139 (+ 46us) server_negotiation.cc:407] Connection header received
0807 00:03:35.545008 (+ 869us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:35.545033 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:35.545496 (+ 463us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:35.545909 (+ 413us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:03:35.546621 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:35.548109 (+ 1488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:35.549009 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:35.549715 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:35.549871 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:35.552746 (+ 2875us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:03:35.552752 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:03:35.552768 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:03:35.552802 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:03:35.554896 (+ 2094us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:35.555358 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:35.555363 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:35.555369 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:35.555513 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:35.555795 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:35.555798 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:35.555800 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:35.556212 (+ 412us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:03:35.556397 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:35.556721 (+ 324us) server_negotiation.cc:299] Negotiation successful
0807 00:03:35.557016 (+ 295us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":303,"thread_start_us":129,"threads_started":1}
I20260807 00:03:35.559576 16764 raft_consensus.cc:348] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42643 } }
I20260807 00:03:35.559815 16764 raft_consensus.cc:374] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:03:35.559890 16764 raft_consensus.cc:729] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ffd7bd1a42094fb7aa13dee38b7d1111, State: Initialized, Role: FOLLOWER
I20260807 00:03:35.560330 16764 consensus_queue.cc:260] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42643 } }
I20260807 00:03:35.560470 16764 raft_consensus.cc:388] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:03:35.560535 16764 raft_consensus.cc:482] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:03:35.560632 16764 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:03:35.561657 16764 raft_consensus.cc:504] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42643 } }
I20260807 00:03:35.561990 16764 leader_election.cc:302] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: ffd7bd1a42094fb7aa13dee38b7d1111; no voters:
I20260807 00:03:35.562316 16764 leader_election.cc:288] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:03:35.562427 16771 raft_consensus.cc:2781] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:03:35.562671 16771 raft_consensus.cc:686] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [term 1 LEADER]: Becoming Leader. State: Replica: ffd7bd1a42094fb7aa13dee38b7d1111, State: Running, Role: LEADER
I20260807 00:03:35.563016 16771 consensus_queue.cc:237] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42643 } }
I20260807 00:03:35.563609 16764 sys_catalog.cc:564] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:03:35.564846 16772 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42643 } } }
I20260807 00:03:35.564841 16773 sys_catalog.cc:455] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [sys.catalog]: SysCatalogTable state changed. Reason: New leader ffd7bd1a42094fb7aa13dee38b7d1111. Latest consensus state: current_term: 1 leader_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "ffd7bd1a42094fb7aa13dee38b7d1111" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42643 } } }
I20260807 00:03:35.565021 16773 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [sys.catalog]: This master's current role is: LEADER
I20260807 00:03:35.565325 16772 sys_catalog.cc:458] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111 [sys.catalog]: This master's current role is: LEADER
I20260807 00:03:35.565956 16764 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 00:03:35.566213 16789 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:03:35.572369 16789 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:03:35.580852 16789 catalog_manager.cc:1347] Generated new cluster ID: 0e60ebb1321d4ba2b5e603fe74710e84
I20260807 00:03:35.581043 16789 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:03:35.598564 16789 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:03:35.600063 16789 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:03:35.617333 16789 catalog_manager.cc:5868] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111: Generated new TSK 0
I20260807 00:03:35.618510 16789 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:03:36.327208 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:47131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:42643
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:03:36.484336 16839 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:03:36.484763 16839 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:03:36.484835 16839 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:03:36.489558 16839 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:03:36.489642 16839 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:03:36.489728 16839 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:03:36.497529 16839 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.204.84:47131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:42643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16839
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:03:36.499102 16839 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:03:36.500576 16839 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:03:36.512920 16853 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:03:36.512928 16839 server_base.cc:1034] running on GCE node
W20260807 00:03:36.512892 16854 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:03:36.512899 16861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:03:36.513752 16839 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:03:36.514456 16839 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:03:36.515616 16839 hybrid_clock.cc:648] HybridClock initialized: now 1786061016515587 us; error 50 us; skew 500 ppm
Aug 07 00:03:36 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:03:36.518566 16839 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:03:36.519623 16839 webserver.cc:460] Webserver started at http://127.30.204.65:37413/ using document root <none> and password file <none>
I20260807 00:03:36.520206 16839 fs_manager.cc:359] Metadata directory not provided
I20260807 00:03:36.520278 16839 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:03:36.520526 16839 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:03:36 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:03:36.537569 16839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "ab759919825045d4b1b0ec01745ab874"
format_stamp: "Formatted at 2026-08-07 00:03:36 on dist-test-slave-pww0"
server_key: "af6250d755ebaff9659ae4e1592ae4c8"
server_key_iv: "b7538be272b2cf383d70130f44354e35"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:36.551905 16839 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "ab759919825045d4b1b0ec01745ab874"
format_stamp: "Formatted at 2026-08-07 00:03:36 on dist-test-slave-pww0"
server_key: "af6250d755ebaff9659ae4e1592ae4c8"
server_key_iv: "b7538be272b2cf383d70130f44354e35"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:36.556758 16839 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:03:36.584088 16874 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:36.586153 16839 fs_manager.cc:725] Time spent opening block manager: real 0.010s user 0.001s sys 0.000s
I20260807 00:03:36.586362 16839 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "ab759919825045d4b1b0ec01745ab874"
format_stamp: "Formatted at 2026-08-07 00:03:36 on dist-test-slave-pww0"
server_key: "af6250d755ebaff9659ae4e1592ae4c8"
server_key_iv: "b7538be272b2cf383d70130f44354e35"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:36.586534 16839 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:03:36.635083 16839 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:03:36.639305 16839 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:03:36.639580 16839 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:03:36.640439 16839 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:03:36.641652 16839 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:03:36.641728 16839 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:36.641793 16839 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:03:36.641942 16839 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.001s sys 0.000s
Aug 07 00:03:36 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:03:36.661064 16839 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:43481
I20260807 00:03:36.661149 17058 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:43481 every 8 connection(s)
I20260807 00:03:36.662581 16839 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:03:36.673075 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 16839
I20260807 00:03:36.673226 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:03:36.673539 31537 external_mini_cluster.cc:1428] Setting key 85487afd7fc185d34fb0cecb7300cee2
Aug 07 00:03:36 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:03:36.681002 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:36.665008 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:42845 (local address 127.30.204.126:42643)
0807 00:03:36.665381 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:36.665388 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:36.666383 (+ 995us) server_negotiation.cc:407] Connection header received
0807 00:03:36.667796 (+ 1413us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:36.667800 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:36.667870 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:36.667983 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:03:36.669668 (+ 1685us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:36.670545 (+ 877us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:36.672479 (+ 1934us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:36.672873 (+ 394us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:36.672977 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:36.676465 (+ 3488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:03:36.676469 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:03:36.676472 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:03:36.676501 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:03:36.678321 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:36.679020 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:36.679024 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:36.679026 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:36.679137 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:36.679594 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:36.679598 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:36.679599 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:36.679837 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:03:36.679916 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:36.680659 (+ 743us) server_negotiation.cc:299] Negotiation successful
0807 00:03:36.680810 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":259,"thread_start_us":110,"threads_started":1}
I20260807 00:03:36.683123 17066 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:36.665574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:42643 (local address 127.30.204.65:42845)
0807 00:03:36.666186 (+ 612us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:36.666253 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:36.667508 (+ 1255us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:36.668220 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:36.668231 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:36.668656 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:03:36.669391 (+ 735us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:36.669416 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:36.670758 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:36.670762 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:36.672292 (+ 1530us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:36.672308 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:36.673260 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:36.673265 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:36.673505 (+ 240us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:36.674291 (+ 786us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:03:36.674320 (+ 29us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:03:36.676250 (+ 1930us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:03:36.678481 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:03:36.678488 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:03:36.678510 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:03:36.678895 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:03:36.679247 (+ 352us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:03:36.679250 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:03:36.679252 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:03:36.679448 (+ 196us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:03:36.679987 (+ 539us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:03:36.679994 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:03:36.680371 (+ 377us) client_negotiation.cc:769] Sending connection context
0807 00:03:36.681509 (+ 1138us) client_negotiation.cc:240] Negotiation successful
0807 00:03:36.681912 (+ 403us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":347,"thread_start_us":149,"threads_started":1}
I20260807 00:03:36.684990 17061 heartbeater.cc:347] Connected to a master server at 127.30.204.126:42643
I20260807 00:03:36.685345 17061 heartbeater.cc:464] Registering TS with master...
I20260807 00:03:36.686048 17061 heartbeater.cc:511] Master 127.30.204.126:42643 requested a full tablet report, sending...
I20260807 00:03:36.687984 16696 ts_manager.cc:194] Registered new tserver with Master: ab759919825045d4b1b0ec01745ab874 (127.30.204.65:43481)
I20260807 00:03:36.689560 16696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:42845
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:03:36.723212 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:47131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:42643
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:03:36.879134 17070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:03:36.879612 17070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:03:36.879755 17070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:03:36.886108 17070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:03:36.886264 17070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:03:36.886371 17070 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:03:36.894752 17070 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.204.84:47131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:42643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:03:36.896569 17070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:03:36.898140 17070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:03:36.912554 17080 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:03:36.913748 17078 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:03:36.914016 17083 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:03:36.914127 17070 server_base.cc:1034] running on GCE node
I20260807 00:03:36.914870 17070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:03:36.915901 17070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:03:36.919265 17070 hybrid_clock.cc:648] HybridClock initialized: now 1786061016919212 us; error 63 us; skew 500 ppm
Aug 07 00:03:36 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:03:36.924938 17070 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:03:36.926540 17070 webserver.cc:460] Webserver started at http://127.30.204.66:42655/ using document root <none> and password file <none>
I20260807 00:03:36.927433 17070 fs_manager.cc:359] Metadata directory not provided
I20260807 00:03:36.927595 17070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:03:36.927956 17070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:03:36 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:03:36.944782 17070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "65b2703a7d9342169a2a1ea343e622d1"
format_stamp: "Formatted at 2026-08-07 00:03:36 on dist-test-slave-pww0"
server_key: "eae853e2f9cc0162e7b960c34c58c601"
server_key_iv: "713280940d9ad4dbc14132e3fb6f8644"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:36.945636 17070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "65b2703a7d9342169a2a1ea343e622d1"
format_stamp: "Formatted at 2026-08-07 00:03:36 on dist-test-slave-pww0"
server_key: "eae853e2f9cc0162e7b960c34c58c601"
server_key_iv: "713280940d9ad4dbc14132e3fb6f8644"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:36.951095 17070 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.002s sys 0.004s
I20260807 00:03:36.980006 17092 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:36.982095 17070 fs_manager.cc:725] Time spent opening block manager: real 0.016s user 0.001s sys 0.000s
I20260807 00:03:36.982304 17070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "65b2703a7d9342169a2a1ea343e622d1"
format_stamp: "Formatted at 2026-08-07 00:03:36 on dist-test-slave-pww0"
server_key: "eae853e2f9cc0162e7b960c34c58c601"
server_key_iv: "713280940d9ad4dbc14132e3fb6f8644"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:36.982488 17070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:03:37.006270 17070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:03:37.010711 17070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:03:37.011013 17070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:03:37.011993 17070 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:03:37.013303 17070 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:03:37.013429 17070 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.001s sys 0.000s
I20260807 00:03:37.013545 17070 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:03:37.013621 17070 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:37.025781 17070 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:40225
I20260807 00:03:37.025816 17267 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:40225 every 8 connection(s)
I20260807 00:03:37.027120 17070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:03:37.031426 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 17070
I20260807 00:03:37.031605 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:03:37.031942 31537 external_mini_cluster.cc:1428] Setting key c0c279c8d3e62b48cd934ae96672ec2b
Aug 07 00:03:37 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:03:37.050340 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.031030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:42643 (local address 127.30.204.66:46251)
0807 00:03:37.032983 (+ 1953us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.033055 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:37.034462 (+ 1407us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:37.035347 (+ 885us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:37.035359 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:37.036103 (+ 744us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:03:37.036756 (+ 653us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.036777 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.037948 (+ 1171us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.037952 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.039402 (+ 1450us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.039417 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.040039 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.040043 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.040250 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.040887 (+ 637us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:03:37.040905 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:03:37.042808 (+ 1903us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:03:37.045200 (+ 2392us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:03:37.045206 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:03:37.045223 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:03:37.045530 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:03:37.045898 (+ 368us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:03:37.045901 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:03:37.045903 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:03:37.046152 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:03:37.047945 (+ 1793us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:03:37.047954 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:03:37.048400 (+ 446us) client_negotiation.cc:769] Sending connection context
0807 00:03:37.048645 (+ 245us) client_negotiation.cc:240] Negotiation successful
0807 00:03:37.049069 (+ 424us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1686,"thread_start_us":184,"threads_started":1}
I20260807 00:03:37.051271 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.030509 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:46251 (local address 127.30.204.126:42643)
0807 00:03:37.030652 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.030657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.033163 (+ 2506us) server_negotiation.cc:407] Connection header received
0807 00:03:37.034753 (+ 1590us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.034757 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.034817 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.034869 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:03:37.036933 (+ 2064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.037766 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.039612 (+ 1846us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.039865 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.039962 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.043071 (+ 3109us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:03:37.043076 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:03:37.043079 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:03:37.043114 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:03:37.045025 (+ 1911us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:37.045658 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:37.045661 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:37.045663 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:37.045773 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:37.046285 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:37.046289 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:37.046291 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:37.046560 (+ 269us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:03:37.046661 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.050998 (+ 4337us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.051148 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:03:37.052672 17271 heartbeater.cc:347] Connected to a master server at 127.30.204.126:42643
I20260807 00:03:37.052958 17271 heartbeater.cc:464] Registering TS with master...
I20260807 00:03:37.053587 17271 heartbeater.cc:511] Master 127.30.204.126:42643 requested a full tablet report, sending...
I20260807 00:03:37.054677 16696 ts_manager.cc:194] Registered new tserver with Master: 65b2703a7d9342169a2a1ea343e622d1 (127.30.204.66:40225)
I20260807 00:03:37.055362 16696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:46251
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:03:37.075690 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:47131/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:42643
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:42181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:03:37 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
W20260807 00:03:37.251408 17280 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:03:37.251937 17280 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:03:37.252043 17280 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:03:37.257771 17280 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:03:37.257855 17280 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:03:37.257941 17280 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:03:37.263758 17280 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:42181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.204.84:47131/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:42643
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17280
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:03:37.265051 17280 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:03:37.266216 17280 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:03:37.276685 17291 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:03:37.276687 17293 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:03:37.276685 17290 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:03:37.277218 17280 server_base.cc:1034] running on GCE node
I20260807 00:03:37.277741 17280 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:03:37.278614 17280 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:03:37.279800 17280 hybrid_clock.cc:648] HybridClock initialized: now 1786061017279782 us; error 30 us; skew 500 ppm
Aug 07 00:03:37 dist-test-slave-pww0 krb5kdc[15909](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061017, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:03:37.283073 17280 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:03:37.284369 17280 webserver.cc:460] Webserver started at http://127.30.204.67:44967/ using document root <none> and password file <none>
I20260807 00:03:37.285005 17280 fs_manager.cc:359] Metadata directory not provided
I20260807 00:03:37.285075 17280 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:03:37.285286 17280 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:03:37 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061017, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:03:37.304661 17280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "967450786cd1404ca17f5470823f1910"
format_stamp: "Formatted at 2026-08-07 00:03:37 on dist-test-slave-pww0"
server_key: "4396b28d632571e2c3bbc4e7d01f1283"
server_key_iv: "e162481218242c79caaa17ea05d55b86"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:37.305639 17280 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "967450786cd1404ca17f5470823f1910"
format_stamp: "Formatted at 2026-08-07 00:03:37 on dist-test-slave-pww0"
server_key: "4396b28d632571e2c3bbc4e7d01f1283"
server_key_iv: "e162481218242c79caaa17ea05d55b86"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:37.311865 17280 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.007s sys 0.000s
I20260807 00:03:37.335007 17303 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:37.336618 17280 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.004s sys 0.000s
I20260807 00:03:37.336778 17280 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "967450786cd1404ca17f5470823f1910"
format_stamp: "Formatted at 2026-08-07 00:03:37 on dist-test-slave-pww0"
server_key: "4396b28d632571e2c3bbc4e7d01f1283"
server_key_iv: "e162481218242c79caaa17ea05d55b86"
server_key_version: "kuduclusterkey@0"
I20260807 00:03:37.336925 17280 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:03:37.360992 17280 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:03:37.364241 17280 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:03:37.364440 17280 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:03:37.365160 17280 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:03:37.366135 17280 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:03:37.366189 17280 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:37.366245 17280 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:03:37.366274 17280 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:03:37.377285 17280 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:44269
I20260807 00:03:37.377312 17505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:44269 every 8 connection(s)
I20260807 00:03:37.378430 17280 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:03:37.379560 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 17280
I20260807 00:03:37.379698 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:03:37.380003 31537 external_mini_cluster.cc:1428] Setting key 69bc98a7490f5bc8e991eecdfa3538a9
Aug 07 00:03:37 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061017, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:03:37.394101 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.382765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:33723 (local address 127.30.204.126:42643)
0807 00:03:37.382882 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.382886 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.382902 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:03:37.383554 (+ 652us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.383558 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.383618 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.383691 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:03:37.384846 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.385414 (+ 568us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.386623 (+ 1209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.386858 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.386962 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.389733 (+ 2771us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:03:37.389736 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:03:37.389738 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:03:37.389767 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:03:37.391144 (+ 1377us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:37.391847 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:37.391851 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:37.391853 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:37.391976 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:37.392507 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:37.392511 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:37.392513 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:37.392764 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:03:37.392850 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.393771 (+ 921us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.393916 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:03:37.395921 17509 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.381702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:42643 (local address 127.30.204.67:33723)
0807 00:03:37.382222 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.382283 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:37.383288 (+ 1005us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:37.383877 (+ 589us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:37.383884 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:37.384171 (+ 287us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:03:37.384693 (+ 522us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.384708 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.385559 (+ 851us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.385563 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.386486 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.386498 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.387027 (+ 529us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.387031 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.387238 (+ 207us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.387964 (+ 726us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:03:37.387986 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:03:37.389537 (+ 1551us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:03:37.391273 (+ 1736us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:03:37.391280 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:03:37.391300 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:03:37.391715 (+ 415us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:03:37.392157 (+ 442us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:03:37.392160 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:03:37.392162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:03:37.392386 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:03:37.392924 (+ 538us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:03:37.392931 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:03:37.393452 (+ 521us) client_negotiation.cc:769] Sending connection context
0807 00:03:37.394294 (+ 842us) client_negotiation.cc:240] Negotiation successful
0807 00:03:37.394738 (+ 444us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":117,"threads_started":1}
I20260807 00:03:37.397400 17506 heartbeater.cc:347] Connected to a master server at 127.30.204.126:42643
I20260807 00:03:37.397706 17506 heartbeater.cc:464] Registering TS with master...
I20260807 00:03:37.398213 17506 heartbeater.cc:511] Master 127.30.204.126:42643 requested a full tablet report, sending...
I20260807 00:03:37.399267 16696 ts_manager.cc:194] Registered new tserver with Master: 967450786cd1404ca17f5470823f1910 (127.30.204.67:44269)
I20260807 00:03:37.399886 16696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:33723
I20260807 00:03:37.406457 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:03:37.417831 17065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.408724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42536 (local address 127.30.204.126:42643)
0807 00:03:37.408849 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.408854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.409144 (+ 290us) server_negotiation.cc:407] Connection header received
0807 00:03:37.409260 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.409263 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.409312 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.409379 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:03:37.410128 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.410679 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.411552 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.411786 (+ 234us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.411878 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.414374 (+ 2496us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:03:37.414377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:03:37.414379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:03:37.414409 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:03:37.415955 (+ 1546us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:37.416400 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:37.416404 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:37.416405 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:37.416491 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:03:37.416769 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:03:37.416772 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:03:37.416774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:03:37.416977 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:03:37.417112 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.417361 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.417490 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:03:37.420969 16696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42536:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:03:37.425347 16696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:03:37.443122 17528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.435008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:44269 (local address 127.0.0.1:34604)
0807 00:03:37.436079 (+ 1071us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.436133 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:37.436264 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:37.438562 (+ 2298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:37.438569 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:37.438591 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:03:37.438736 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.438742 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.440351 (+ 1609us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.440355 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.441465 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.441476 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.442549 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.442552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.442677 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.442721 (+ 44us) client_negotiation.cc:769] Sending connection context
0807 00:03:37.442825 (+ 104us) client_negotiation.cc:240] Negotiation successful
0807 00:03:37.442968 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":877,"thread_start_us":764,"threads_started":1}
I20260807 00:03:37.443146 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.435429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34604 (local address 127.30.204.67:44269)
0807 00:03:37.438038 (+ 2609us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.438051 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.438079 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:03:37.438183 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.438190 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.438410 (+ 220us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.438561 (+ 151us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:03:37.438851 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.439971 (+ 1120us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.441613 (+ 1642us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.442424 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.442529 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.442695 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.442873 (+ 178us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.442975 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2492,"thread_start_us":119,"threads_started":1}
I20260807 00:03:37.446969 17366 tablet_service.cc:1467] Processing CreateTablet for tablet 8e568f44b9a643459792bd6fb55132bc (DEFAULT_TABLE table=test-table [id=c25857dda70842199939bd3803d9c60c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:03:37.449812 17366 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e568f44b9a643459792bd6fb55132bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:03:37.453558 17529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.436898 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:40225 (local address 127.0.0.1:52106)
0807 00:03:37.437240 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.438134 (+ 894us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:37.438220 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:37.445424 (+ 7204us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:37.445427 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:37.445447 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:03:37.445546 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.445552 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.448366 (+ 2814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.448369 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.449443 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.449455 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.453080 (+ 3625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.453084 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.453211 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.453239 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 00:03:37.453311 (+ 72us) client_negotiation.cc:240] Negotiation successful
0807 00:03:37.453396 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":102,"threads_started":1}
I20260807 00:03:37.454321 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.437128 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:43481 (local address 127.0.0.1:58940)
0807 00:03:37.437665 (+ 537us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.437678 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:37.437769 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:37.444948 (+ 7179us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:37.444951 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:37.444976 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:03:37.445080 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.445086 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.449601 (+ 4515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.449604 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.450708 (+ 1104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.450719 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.453904 (+ 3185us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.453908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.454007 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.454035 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 00:03:37.454085 (+ 50us) client_negotiation.cc:240] Negotiation successful
0807 00:03:37.454172 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":470,"thread_start_us":98,"threads_started":1}
I20260807 00:03:37.455168 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.437609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58940 (local address 127.30.204.65:43481)
0807 00:03:37.443712 (+ 6103us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.443720 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.443746 (+ 26us) server_negotiation.cc:407] Connection header received
0807 00:03:37.443825 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.443831 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.444056 (+ 225us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.445727 (+ 1671us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:03:37.445883 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.446961 (+ 1078us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.450883 (+ 3922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.451758 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.454635 (+ 2877us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.454824 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.454887 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.455003 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5974,"thread_start_us":93,"threads_started":1}
I20260807 00:03:37.456507 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.437077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52106 (local address 127.30.204.66:40225)
0807 00:03:37.444397 (+ 7320us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.444409 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.444434 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:03:37.444513 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.444518 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.444724 (+ 206us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.445276 (+ 552us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:03:37.447115 (+ 1839us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.448239 (+ 1124us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.451925 (+ 3686us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.452735 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.452841 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.453011 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.456227 (+ 3216us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.456361 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":7199,"thread_start_us":128,"threads_started":1}
I20260807 00:03:37.458787 16947 tablet_service.cc:1467] Processing CreateTablet for tablet 8e568f44b9a643459792bd6fb55132bc (DEFAULT_TABLE table=test-table [id=c25857dda70842199939bd3803d9c60c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:03:37.460196 16947 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e568f44b9a643459792bd6fb55132bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:03:37.460830 17533 tablet_bootstrap.cc:492] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Bootstrap starting.
I20260807 00:03:37.463824 17533 tablet_bootstrap.cc:654] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Neither blocks nor log segments found. Creating new log.
I20260807 00:03:37.464885 17533 log.cc:824] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Log is configured to *not* fsync() on all Append() calls
I20260807 00:03:37.466076 17171 tablet_service.cc:1467] Processing CreateTablet for tablet 8e568f44b9a643459792bd6fb55132bc (DEFAULT_TABLE table=test-table [id=c25857dda70842199939bd3803d9c60c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:03:37.466992 17533 tablet_bootstrap.cc:492] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: No bootstrap required, opened a new log
I20260807 00:03:37.467231 17533 ts_tablet_manager.cc:1397] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:03:37.467350 17171 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e568f44b9a643459792bd6fb55132bc. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:03:37.471407 17533 raft_consensus.cc:348] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.471771 17533 raft_consensus.cc:374] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:03:37.471835 17533 raft_consensus.cc:729] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 967450786cd1404ca17f5470823f1910, State: Initialized, Role: FOLLOWER
I20260807 00:03:37.472450 17533 consensus_queue.cc:260] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.473289 17506 heartbeater.cc:503] Master 127.30.204.126:42643 was elected leader, sending a full tablet report...
I20260807 00:03:37.473608 17533 ts_tablet_manager.cc:1428] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 00:03:37.475098 17539 tablet_bootstrap.cc:492] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Bootstrap starting.
I20260807 00:03:37.478300 17539 tablet_bootstrap.cc:654] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Neither blocks nor log segments found. Creating new log.
I20260807 00:03:37.479249 17539 log.cc:824] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Log is configured to *not* fsync() on all Append() calls
I20260807 00:03:37.481678 17539 tablet_bootstrap.cc:492] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: No bootstrap required, opened a new log
I20260807 00:03:37.481913 17539 ts_tablet_manager.cc:1397] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 00:03:37.480948 17542 tablet_bootstrap.cc:492] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874: Bootstrap starting.
I20260807 00:03:37.485144 17542 tablet_bootstrap.cc:654] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874: Neither blocks nor log segments found. Creating new log.
I20260807 00:03:37.486195 17542 log.cc:824] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874: Log is configured to *not* fsync() on all Append() calls
I20260807 00:03:37.486611 17539 raft_consensus.cc:348] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.487004 17539 raft_consensus.cc:374] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:03:37.487102 17539 raft_consensus.cc:729] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65b2703a7d9342169a2a1ea343e622d1, State: Initialized, Role: FOLLOWER
I20260807 00:03:37.487821 17539 consensus_queue.cc:260] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.490710 17539 ts_tablet_manager.cc:1428] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Time spent starting tablet: real 0.009s user 0.007s sys 0.000s
I20260807 00:03:37.491191 17271 heartbeater.cc:503] Master 127.30.204.126:42643 was elected leader, sending a full tablet report...
I20260807 00:03:37.492420 17542 tablet_bootstrap.cc:492] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874: No bootstrap required, opened a new log
I20260807 00:03:37.492647 17542 ts_tablet_manager.cc:1397] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874: Time spent bootstrapping tablet: real 0.012s user 0.004s sys 0.001s
I20260807 00:03:37.497530 17542 raft_consensus.cc:348] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.497895 17542 raft_consensus.cc:374] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:03:37.497963 17542 raft_consensus.cc:729] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ab759919825045d4b1b0ec01745ab874, State: Initialized, Role: FOLLOWER
I20260807 00:03:37.498562 17542 consensus_queue.cc:260] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.499325 17061 heartbeater.cc:503] Master 127.30.204.126:42643 was elected leader, sending a full tablet report...
I20260807 00:03:37.499819 17542 ts_tablet_manager.cc:1428] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874: Time spent starting tablet: real 0.007s user 0.005s sys 0.003s
Aug 07 00:03:37 dist-test-slave-pww0 krb5kdc[15909](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061016, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:03:37.524655 17547 raft_consensus.cc:482] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:03:37.524896 17547 raft_consensus.cc:504] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.526158 17547 leader_election.cc:288] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 967450786cd1404ca17f5470823f1910 (127.30.204.67:44269), ab759919825045d4b1b0ec01745ab874 (127.30.204.65:43481)
W20260807 00:03:37.529381 17272 tablet.cc:2369] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:03:37.531239 17273 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.526453 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:43481 (local address 127.30.204.66:52127)
0807 00:03:37.526870 (+ 417us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.526881 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:37.526978 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:37.527503 (+ 525us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:37.527506 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:37.527526 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:03:37.527677 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.527683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.528649 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.528653 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.529775 (+ 1122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.529786 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.530774 (+ 988us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.530777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.530878 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.530902 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:03:37.530972 (+ 70us) client_negotiation.cc:240] Negotiation successful
0807 00:03:37.531056 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":3,"spinlock_wait_cycles":19840,"thread_start_us":144,"threads_started":1}
I20260807 00:03:37.531677 17532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.527137 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:52127 (local address 127.30.204.65:43481)
0807 00:03:37.527253 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.527258 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.527272 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:03:37.527327 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.527329 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.527378 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.527492 (+ 114us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:03:37.527802 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.528506 (+ 704us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.529949 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.530530 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.530648 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.530707 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.531445 (+ 738us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.531561 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:03:37.532298 17527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.526740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:33339 (local address 127.30.204.67:44269)
0807 00:03:37.526862 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.526866 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.526885 (+ 19us) server_negotiation.cc:407] Connection header received
0807 00:03:37.527081 (+ 196us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.527085 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.527146 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.527240 (+ 94us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:03:37.527799 (+ 559us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.528596 (+ 797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.530378 (+ 1782us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.531018 (+ 640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.531115 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.531165 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.532096 (+ 931us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.532180 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 00:03:37.532311 16977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e568f44b9a643459792bd6fb55132bc" candidate_uuid: "65b2703a7d9342169a2a1ea343e622d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab759919825045d4b1b0ec01745ab874" is_pre_election: true
I20260807 00:03:37.532670 16977 raft_consensus.cc:2454] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b2703a7d9342169a2a1ea343e622d1 in term 0.
I20260807 00:03:37.532982 17552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.526388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:44269 (local address 127.30.204.66:33339)
0807 00:03:37.526817 (+ 429us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.526833 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:03:37.526947 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:03:37.527387 (+ 440us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:03:37.527390 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:03:37.527415 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:03:37.527637 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.527645 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.528792 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.528795 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.530195 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:03:37.530209 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.531283 (+ 1074us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.531287 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:03:37.531937 (+ 650us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.531964 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:03:37.532655 (+ 691us) client_negotiation.cc:240] Negotiation successful
0807 00:03:37.532833 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"mutex_wait_us":48}
I20260807 00:03:37.533888 17415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e568f44b9a643459792bd6fb55132bc" candidate_uuid: "65b2703a7d9342169a2a1ea343e622d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "967450786cd1404ca17f5470823f1910" is_pre_election: true
I20260807 00:03:37.534240 17415 raft_consensus.cc:2454] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 65b2703a7d9342169a2a1ea343e622d1 in term 0.
I20260807 00:03:37.534814 17093 leader_election.cc:302] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65b2703a7d9342169a2a1ea343e622d1, 967450786cd1404ca17f5470823f1910; no voters:
I20260807 00:03:37.535125 17547 raft_consensus.cc:2781] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:03:37.535218 17547 raft_consensus.cc:482] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:03:37.535251 17547 raft_consensus.cc:3037] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:03:37.536505 17547 raft_consensus.cc:504] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.536886 17547 leader_election.cc:288] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [CANDIDATE]: Term 1 election: Requested vote from peers 967450786cd1404ca17f5470823f1910 (127.30.204.67:44269), ab759919825045d4b1b0ec01745ab874 (127.30.204.65:43481)
I20260807 00:03:37.537307 16977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e568f44b9a643459792bd6fb55132bc" candidate_uuid: "65b2703a7d9342169a2a1ea343e622d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ab759919825045d4b1b0ec01745ab874"
I20260807 00:03:37.537307 17415 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e568f44b9a643459792bd6fb55132bc" candidate_uuid: "65b2703a7d9342169a2a1ea343e622d1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "967450786cd1404ca17f5470823f1910"
I20260807 00:03:37.537449 16977 raft_consensus.cc:3037] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:03:37.537449 17415 raft_consensus.cc:3037] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:03:37.538779 16977 raft_consensus.cc:2454] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b2703a7d9342169a2a1ea343e622d1 in term 1.
I20260807 00:03:37.538779 17415 raft_consensus.cc:2454] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 65b2703a7d9342169a2a1ea343e622d1 in term 1.
I20260807 00:03:37.539178 17093 leader_election.cc:302] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 65b2703a7d9342169a2a1ea343e622d1, 967450786cd1404ca17f5470823f1910; no voters:
I20260807 00:03:37.543923 17547 raft_consensus.cc:2781] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:03:37.544207 17547 raft_consensus.cc:686] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 1 LEADER]: Becoming Leader. State: Replica: 65b2703a7d9342169a2a1ea343e622d1, State: Running, Role: LEADER
I20260807 00:03:37.544659 17547 consensus_queue.cc:237] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } }
I20260807 00:03:37.559372 16696 catalog_manager.cc:5495] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 65b2703a7d9342169a2a1ea343e622d1 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "65b2703a7d9342169a2a1ea343e622d1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "65b2703a7d9342169a2a1ea343e622d1" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40225 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 } health_report { overall_health: UNKNOWN } } }
I20260807 00:03:37.577236 17530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.571591 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52112 (local address 127.30.204.66:40225)
0807 00:03:37.571912 (+ 321us) server_negotiation.cc:206] Beginning negotiation
0807 00:03:37.571917 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:03:37.571931 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:03:37.572031 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:03:37.572034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:03:37.572107 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:03:37.572233 (+ 126us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:03:37.572662 (+ 429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.573446 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.574694 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:03:37.575024 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:03:37.575143 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:03:37.575676 (+ 533us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:03:37.576314 (+ 638us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:03:37.576436 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 00:03:37.576952 (+ 516us) server_negotiation.cc:299] Negotiation successful
0807 00:03:37.577030 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":222}
I20260807 00:03:37.586004 16977 raft_consensus.cc:1261] T 8e568f44b9a643459792bd6fb55132bc P ab759919825045d4b1b0ec01745ab874 [term 1 FOLLOWER]: Refusing update from remote peer 65b2703a7d9342169a2a1ea343e622d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:03:37.586084 17415 raft_consensus.cc:1261] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 1 FOLLOWER]: Refusing update from remote peer 65b2703a7d9342169a2a1ea343e622d1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:03:37.586885 17547 consensus_queue.cc:1035] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "967450786cd1404ca17f5470823f1910" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 44269 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:03:37.587436 17553 consensus_queue.cc:1035] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "ab759919825045d4b1b0ec01745ab874" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 43481 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:03:37.603044 17558 mvcc.cc:205] Tried to move back new op lower bound from 7315705928022056960 to 7315705927910006784. Current Snapshot: MvccSnapshot[applied={T|T < 7315705928022056960 or (T in {7315705928022056960})}]
I20260807 00:03:37.603482 17561 mvcc.cc:205] Tried to move back new op lower bound from 7315705928022056960 to 7315705927910006784. Current Snapshot: MvccSnapshot[applied={T|T < 7315705928022056960}]
I20260807 00:03:37.611799 16702 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:03:37.612027 16702 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:42536:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:03:37.620700 16947 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e568f44b9a643459792bd6fb55132bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:03:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58940
I20260807 00:03:37.622367 17171 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e568f44b9a643459792bd6fb55132bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:03:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:52106
I20260807 00:03:37.623138 16702 catalog_manager.cc:5782] T 00000000000000000000000000000000 P ffd7bd1a42094fb7aa13dee38b7d1111: Sending DeleteTablet for 3 replicas of tablet 8e568f44b9a643459792bd6fb55132bc
I20260807 00:03:37.624085 17366 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e568f44b9a643459792bd6fb55132bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:03:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:34604
I20260807 00:03:37.624548 17171 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e568f44b9a643459792bd6fb55132bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:03:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:52106
I20260807 00:03:37.624750 16947 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e568f44b9a643459792bd6fb55132bc with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:03:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58940
I20260807 00:03:37.625687 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 16839
I20260807 00:03:37.645161 17570 tablet_replica.cc:330] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: stopping tablet replica
I20260807 00:03:37.645682 17570 raft_consensus.cc:2229] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:03:37.646200 17570 raft_consensus.cc:2258] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:03:37.651288 17571 tablet_replica.cc:330] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: stopping tablet replica
I20260807 00:03:37.651755 17571 raft_consensus.cc:2229] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:03:37.652164 17571 raft_consensus.cc:2258] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:03:37.654150 17571 ts_tablet_manager.cc:1905] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:03:37.654671 17570 ts_tablet_manager.cc:1905] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:03:37.657299 17570 ts_tablet_manager.cc:1918] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:03:37.657285 17571 ts_tablet_manager.cc:1918] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:03:37.657407 17570 log.cc:1196] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/8e568f44b9a643459792bd6fb55132bc
I20260807 00:03:37.657407 17571 log.cc:1196] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/8e568f44b9a643459792bd6fb55132bc
I20260807 00:03:37.657733 17570 ts_tablet_manager.cc:1939] T 8e568f44b9a643459792bd6fb55132bc P 65b2703a7d9342169a2a1ea343e622d1: Deleting consensus metadata
I20260807 00:03:37.657733 17571 ts_tablet_manager.cc:1939] T 8e568f44b9a643459792bd6fb55132bc P 967450786cd1404ca17f5470823f1910: Deleting consensus metadata
I20260807 00:03:37.659101 16680 catalog_manager.cc:4841] TS 65b2703a7d9342169a2a1ea343e622d1 (127.30.204.66:40225): tablet 8e568f44b9a643459792bd6fb55132bc (table test-table [id=c25857dda70842199939bd3803d9c60c]) successfully deleted
I20260807 00:03:37.659101 16675 catalog_manager.cc:4841] TS 967450786cd1404ca17f5470823f1910 (127.30.204.67:44269): tablet 8e568f44b9a643459792bd6fb55132bc (table test-table [id=c25857dda70842199939bd3803d9c60c]) successfully deleted
W20260807 00:03:37.659615 16678 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:43481 (error 108)
I20260807 00:03:37.659920 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 17070
I20260807 00:03:37.674211 17531 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:03:37.673737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:43481 (local address 127.0.0.1:58948)
0807 00:03:37.674004 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.674100 (+ 96us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:43481: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":178}
W20260807 00:03:37.674474 16678 catalog_manager.cc:4584] TS ab759919825045d4b1b0ec01745ab874 (127.30.204.65:43481): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e568f44b9a643459792bd6fb55132bc: Network error: Client connection negotiation failed: client connection to 127.30.204.65:43481: connect: Connection refused (error 111)
W20260807 00:03:37.681808 17531 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:03:37.681403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:40225 (local address 127.0.0.1:52114)
0807 00:03:37.681576 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0807 00:03:37.681661 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.66:40225: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":51}
I20260807 00:03:37.682056 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 17280
I20260807 00:03:37.695446 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 16646
I20260807 00:03:37.717972 31537 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 00:03:38.064081 31537 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 00:03:38.064337 31537 mini_ranger.cc:66] Stopping Ranger...
I20260807 00:03:38.410102 31537 mini_ranger.cc:68] Stopped Ranger
2026-08-07 00:03:38.410 UTC [15954] LOG: received smart shutdown request
2026-08-07 00:03:38.414 UTC [15954] LOG: background worker "logical replication launcher" (PID 15968) exited with exit code 1
2026-08-07 00:03:38.414 UTC [15959] LOG: shutting down
2026-08-07 00:03:38.594 UTC [15954] LOG: database system is shut down
2026-08-07T00:03:38Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (50491 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:03:38 dist-test-slave-pww0 krb5kdc[17589](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:03:38 dist-test-slave-pww0 krb5kdc[17589](info): set up 2 sockets
Aug 07 00:03:38 dist-test-slave-pww0 krb5kdc[17589](info): commencing operation
krb5kdc: starting...
W20260807 00:03:39.857867 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.138s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:03:39 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061019, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:03:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:03:39Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/rangeradmin_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/rangeradmin_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/rangerlookup_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/rangerlookup_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
I20260807 00:03:40.051183 31537 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskl_Zz3S/build/debug/bin/postgres/pg_ctl -D /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/postgres -l logfile start
2026-08-07 00:03:42.602 UTC [17649] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-08-07 00:03:42.602 UTC [17649] LOG: listening on IPv4 address "127.30.204.84", port 57431
2026-08-07 00:03:42.605 UTC [17649] LOG: listening on Unix socket "/tmp/.s.PGSQL.57431"
2026-08-07 00:03:42.626 UTC [17653] LOG: database system was shut down at 2026-08-07 00:03:41 UTC
2026-08-07 00:03:42.630 UTC [17649] LOG: database system is ready to accept connections
I20260807 00:03:43.695796 31537 mini_postgres.cc:96] Postgres bound to 57431
2026-08-07 00:03:43.703 UTC [17666] FATAL: database "slave" does not exist
127.30.204.84:57431 - accepting connections
I20260807 00:03:43.704068 31537 mini_ranger.cc:161] Starting Ranger...
I20260807 00:03:43.720793 31537 mini_ranger.cc:84] Created miniranger Postgres user
I20260807 00:03:44.206951 31537 mini_ranger.cc:87] Created ranger Postgres database
I20260807 00:03:44.207028 31537 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-admin
2026-08-07 00:03:44,484 [I] DB FLAVOR :POSTGRES
2026-08-07 00:03:44,484 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:03:44,484 [I] Checking connection..
2026-08-07 00:03:44,484 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-08-07 00:03:44,762 [I] Checking connection passed.
2026-08-07 00:03:44,762 [I] --------- Verifying version history table ---------
2026-08-07 00:03:44,762 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:03:45,045 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:03:45,046 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:03:45,327 [I] Table x_db_version_h does not exist in database ranger
2026-08-07 00:03:45,327 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-08-07 00:03:45,327 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-08-07 00:03:45.588 UTC [17765] WARNING: there is no transaction in progress
2026-08-07 00:03:45,597 [I] create_dbversion_catalog.sql file imported successfully
2026-08-07 00:03:45,598 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-08-07 00:03:45,875 [I] Table x_db_version_h already exists in database 'ranger'
2026-08-07 00:03:45,875 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:03:45,875 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-08-07 00:03:46,138 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-08-07 00:03:46,413 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-pww0.c.gcp-upstream.internal','N') ;"
2026-08-07 00:03:46,677 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:03:46,932 [I] Table x_portal_user does not exist in database ranger
2026-08-07 00:03:46,932 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:03:47,218 [I] Table x_policy_ref_group does not exist in database ranger
2026-08-07 00:03:47,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:03:47,477 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:03:47,747 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-08-07 00:03:47,747 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-08-07 00:03:48.796 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.814 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.833 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.850 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.867 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.887 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.950 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.959 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.975 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.986 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:48.998 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:49.015 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:49.026 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:49.033 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:49.041 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:49.049 UTC [17943] WARNING: there is no transaction in progress
2026-08-07 00:03:49,796 [I] ranger_core_db_postgres.sql file imported successfully
2026-08-07 00:03:49,796 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-08-07 00:03:50,056 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-08-07 00:03:50,056 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-08-07 00:03:50,318 [I] Table x_portal_user already exists in database 'ranger'
2026-08-07 00:03:50,318 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-08-07 00:03:50,574 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-08-07 00:03:50,574 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-08-07 00:03:50,838 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-08-07 00:03:51,110 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-pww0.c.gcp-upstream.internal';"
2026-08-07 00:03:51,362 [I] CORE_DB_SCHEMA import status has been updated
2026-08-07 00:03:51,362 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-08-07 00:03:51,620 [I] DB_PATCHES have already been applied
I20260807 00:03:51.626626 31537 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-admin:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskl_Zz3S/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260807 00:03:51.626705 31537 mini_ranger.cc:193] Using host: 127.30.204.84
I20260807 00:03:51.632871 31537 mini_ranger.cc:227] Ranger admin URL: http://127.30.204.84:56509
Aug 07, 2026 12:03:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:03:51 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp
Aug 07, 2026 12:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Aug 07, 2026 12:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp] .....
Aug 07, 2026 12:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-admin/ews/webapp].
Aug 07, 2026 12:03:52 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260807 00:03:52.892628 31537 mini_ranger.cc:160] Time spent starting Ranger: real 9.189s user 0.002s sys 0.011s
Aug 07, 2026 12:04:13 AM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Aug 07, 2026 12:04:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
Aug 07, 2026 12:04:13 AM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Aug 07, 2026 12:04:13 AM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Aug 07, 2026 12:04:14 AM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260807 00:04:14.963655 31537 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.30.204.84@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.30.204.84@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/rangerkms_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.30.204.84@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/rangerkms_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.30.204.84@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.30.204.84@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
Entry for principal HTTP/127.30.204.84@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/HTTP_127.30.204.84@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Aug 07 00:04:15 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061055, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260807 00:04:15.060845 31537 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260807 00:04:15.073530 31537 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260807 00:04:16.010509 31537 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260807 00:04:16.010602 31537 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms
I20260807 00:04:16.010623 31537 mini_ranger_kms.cc:227] Using postgres at 127.30.204.84:57431
2026-08-07 00:04:16,188 [I] DB FLAVOR :POSTGRES
2026-08-07 00:04:16,188 [I] --------- Verifying Ranger DB connection ---------
2026-08-07 00:04:16,188 [I] Checking connection
2026-08-07 00:04:16,188 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-08-07 00:04:16,444 [I] connection success
2026-08-07 00:04:16,444 [I] --------- Verifying Ranger DB tables ---------
2026-08-07 00:04:16,444 [I] Verifying table ranger_masterkey in database rangerkms
2026-08-07 00:04:16,444 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-08-07 00:04:16,706 [I] Table ranger_masterkey does not exist in database rangerkms
2026-08-07 00:04:16,706 [I] --------- Importing Ranger Core DB Schema ---------
2026-08-07 00:04:16,706 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-08-07 00:04:16,707 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.30.204.84:57431/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-08-07 00:04:16,979 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260807 00:04:17.488240 31537 mini_ranger_kms.cc:310] Created kms service
I20260807 00:04:17.685101 31537 mini_ranger_kms.cc:326] Created kudu user
I20260807 00:04:17.735865 31537 mini_ranger_kms.cc:343] Created rangerkms user
Aug 07 00:04:17 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061055, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:18.161520 31537 mini_ranger_kms.cc:384] Added ranger policy
I20260807 00:04:18.161814 31537 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms:/tmp/dist-test-taskl_Zz3S/build/debug/bin/postgresql.jar:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskl_Zz3S/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260807 00:04:18.161844 31537 mini_ranger_kms.cc:240] Using host: 127.30.204.84
I20260807 00:04:18.167893 31537 mini_ranger_kms.cc:276] Ranger KMS PID: 18227
I20260807 00:04:18.167992 31537 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.30.204.84:46075
Aug 07, 2026 12:04:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Aug 07, 2026 12:04:19 AM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Aug 07, 2026 12:04:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Aug 07, 2026 12:04:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms/ews/webapp] .....
Aug 07, 2026 12:04:19 AM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-kms/ews/webapp].
Aug 07, 2026 12:04:19 AM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Aug 07 00:04:20 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786061060, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.204.84@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:20.721920 31537 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.661s user 0.019s sys 0.144s
I20260807 00:04:20.722061 31537 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260807 00:04:20.722136 31537 mini_ranger_kms.cc:401] 127.30.204.84:46075/kms/v1/keys
Aug 07 00:04:21 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061060, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.204.84@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
Aug 07 00:04:21 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061060, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.30.204.84@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
Aug 07 00:04:23 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:04:23.907593 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40641
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:40641
--ranger_config_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:24.032967 18304 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:24.033291 18304 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:24.033332 18304 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:24.037635 18304 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:24.037715 18304 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:24.037734 18304 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:24.037748 18304 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:24.037762 18304 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:24.043709 18304 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
--ranger_kms_url=127.30.204.84:46075/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:40641
--ranger_config_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18304
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:24.044978 18304 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:24.046079 18304 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:24.055677 18309 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:24.055776 18313 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:24.055697 18310 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:24.056178 18304 server_base.cc:1034] running on GCE node
I20260807 00:04:24.056802 18304 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:24.058054 18304 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:24.059252 18304 hybrid_clock.cc:648] HybridClock initialized: now 1786061064059233 us; error 47 us; skew 500 ppm
Aug 07 00:04:24 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:24.066097 18304 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:24.067572 18304 webserver.cc:460] Webserver started at http://127.30.204.126:39899/ using document root <none> and password file <none>
I20260807 00:04:24.068387 18304 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:24.068490 18304 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:24.068747 18304 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:04:24 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061064, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:24.121567 18304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "4711c8f0a3e34924b81e9092438643fe"
format_stamp: "Formatted at 2026-08-07 00:04:24 on dist-test-slave-pww0"
server_key: "7986ae9b27926ff2a980e161fdb88b80"
server_key_iv: "095f06045da7d3028a903c9a897a6ec5"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:24.122596 18304 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "4711c8f0a3e34924b81e9092438643fe"
format_stamp: "Formatted at 2026-08-07 00:04:24 on dist-test-slave-pww0"
server_key: "7986ae9b27926ff2a980e161fdb88b80"
server_key_iv: "095f06045da7d3028a903c9a897a6ec5"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:24.127425 18304 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.003s sys 0.000s
I20260807 00:04:24.143877 18326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:24.145368 18304 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 00:04:24.145547 18304 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "4711c8f0a3e34924b81e9092438643fe"
format_stamp: "Formatted at 2026-08-07 00:04:24 on dist-test-slave-pww0"
server_key: "7986ae9b27926ff2a980e161fdb88b80"
server_key_iv: "095f06045da7d3028a903c9a897a6ec5"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:24.145704 18304 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:24.171159 18304 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:24.174341 18304 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:24.174561 18304 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:24.182868 18304 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:40641
I20260807 00:04:24.182886 18408 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:40641 every 8 connection(s)
I20260807 00:04:24.184010 18304 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:24.187198 18409 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:24.187702 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 18304
I20260807 00:04:24.187834 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:04:24.188133 31537 external_mini_cluster.cc:1428] Setting key 53ac84b10db845d883aacb4bd792a1aa
I20260807 00:04:24.193677 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Bootstrap starting.
I20260807 00:04:24.196666 18409 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:24.197736 18409 log.cc:824] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Log is configured to *not* fsync() on all Append() calls
Aug 07 00:04:24 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061063, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:24.200182 18409 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: No bootstrap required, opened a new log
I20260807 00:04:24.203419 18409 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:24.203750 18409 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:24.203836 18409 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4711c8f0a3e34924b81e9092438643fe, State: Initialized, Role: FOLLOWER
I20260807 00:04:24.204329 18409 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:24.204392 18415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:24.189619 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46128 (local address 127.30.204.126:40641)
0807 00:04:24.190095 (+ 476us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:24.190117 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:24.190331 (+ 214us) server_negotiation.cc:407] Connection header received
0807 00:04:24.191124 (+ 793us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:24.191143 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:24.191554 (+ 411us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:24.191899 (+ 345us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:24.192750 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:24.193794 (+ 1044us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:24.195090 (+ 1296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:24.195662 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:24.195801 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:24.198843 (+ 3042us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:24.198849 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:24.198866 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:24.198897 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:24.201178 (+ 2281us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:24.201663 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:24.201668 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:24.201674 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:24.201802 (+ 128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:24.202097 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:24.202100 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:24.202101 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:24.202548 (+ 447us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:24.202771 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:24.203063 (+ 292us) server_negotiation.cc:299] Negotiation successful
0807 00:04:24.203430 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":111,"threads_started":1}
I20260807 00:04:24.204496 18409 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:04:24.204563 18409 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:04:24.204670 18409 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:24.205947 18409 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:24.206308 18409 leader_election.cc:302] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4711c8f0a3e34924b81e9092438643fe; no voters:
I20260807 00:04:24.206660 18409 leader_election.cc:288] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:04:24.206779 18417 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:24.207003 18417 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 LEADER]: Becoming Leader. State: Replica: 4711c8f0a3e34924b81e9092438643fe, State: Running, Role: LEADER
I20260807 00:04:24.207337 18417 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:24.208002 18409 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:24.209189 18421 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4711c8f0a3e34924b81e9092438643fe. Latest consensus state: current_term: 1 leader_uuid: "4711c8f0a3e34924b81e9092438643fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } } }
I20260807 00:04:24.209370 18421 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:24.209189 18418 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4711c8f0a3e34924b81e9092438643fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } } }
I20260807 00:04:24.209497 18418 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:24.209842 18428 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:24.213466 18428 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:24.214027 18409 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 00:04:24.224108 18428 catalog_manager.cc:1347] Generated new cluster ID: e6cbac0486c34e34b2d62839e793299f
I20260807 00:04:24.224224 18428 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:04:24.238965 18428 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:04:24.240100 18428 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:04:24.245378 18428 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Generated new TSK 0
I20260807 00:04:24.246271 18428 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:04:24.993171 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40641
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:25.171010 18477 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:25.171536 18477 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:25.171695 18477 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:25.178445 18477 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:25.178553 18477 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:25.178660 18477 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:04:25.187860 18477 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.30.204.84:46075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18477
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:25.189530 18477 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:25.191143 18477 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:25.205628 18500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:25.205957 18493 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:25.206148 18491 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:25.206496 18477 server_base.cc:1034] running on GCE node
I20260807 00:04:25.207021 18477 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:25.207960 18477 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:25.209139 18477 hybrid_clock.cc:648] HybridClock initialized: now 1786061065209107 us; error 47 us; skew 500 ppm
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:25.212561 18477 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:04:25.213953 18477 webserver.cc:460] Webserver started at http://127.30.204.65:35275/ using document root <none> and password file <none>
I20260807 00:04:25.214795 18477 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:25.214902 18477 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:25.215155 18477 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:25.233270 18477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "ac8e3304d4ee4e678331b663425cb9cb"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "dd49715a0195b8b809c38642c6ca0882"
server_key_iv: "bd6f96f91205763d3a653d999f6994fb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:25.234229 18477 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "ac8e3304d4ee4e678331b663425cb9cb"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "dd49715a0195b8b809c38642c6ca0882"
server_key_iv: "bd6f96f91205763d3a653d999f6994fb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:25.240526 18477 fs_manager.cc:691] Time spent creating directory manager: real 0.006s user 0.003s sys 0.005s
I20260807 00:04:25.272320 18514 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:25.274435 18477 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.003s sys 0.002s
I20260807 00:04:25.274634 18477 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "ac8e3304d4ee4e678331b663425cb9cb"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "dd49715a0195b8b809c38642c6ca0882"
server_key_iv: "bd6f96f91205763d3a653d999f6994fb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:25.274799 18477 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:25.299571 18477 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:25.303791 18477 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:25.304023 18477 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:25.304865 18477 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:25.306110 18477 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:25.306182 18477 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:25.306250 18477 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:25.306278 18477 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:25.318195 18477 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:36561
I20260807 00:04:25.319705 18477 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:04:25.320354 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 18477
I20260807 00:04:25.320487 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:04:25.320855 31537 external_mini_cluster.cc:1428] Setting key f7635b702bbf929223e9ac68ece022a8
I20260807 00:04:25.329013 18707 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:36561 every 8 connection(s)
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:25.362803 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:25.333667 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:40017 (local address 127.30.204.126:40641)
0807 00:04:25.339746 (+ 6079us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:25.339753 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:25.345642 (+ 5889us) server_negotiation.cc:407] Connection header received
0807 00:04:25.345722 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:25.345726 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:25.345796 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:25.345922 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:25.347911 (+ 1989us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.348871 (+ 960us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.353865 (+ 4994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.354232 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.354327 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:25.357856 (+ 3529us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:25.357860 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:25.357863 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:25.357897 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:25.359946 (+ 2049us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:25.360737 (+ 791us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:25.360742 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:25.360744 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:25.360854 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:25.361316 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:25.361320 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:25.361321 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:25.361587 (+ 266us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:25.361672 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:25.362444 (+ 772us) server_negotiation.cc:299] Negotiation successful
0807 00:04:25.362613 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5948,"thread_start_us":183,"threads_started":1}
I20260807 00:04:25.364861 18721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:25.334703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40641 (local address 127.30.204.65:40017)
0807 00:04:25.343958 (+ 9255us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:25.344052 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:25.345247 (+ 1195us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:25.346185 (+ 938us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:25.346197 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:25.346605 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:25.347448 (+ 843us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:25.347694 (+ 246us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.349073 (+ 1379us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.349077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:25.353641 (+ 4564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:25.353666 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.354612 (+ 946us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.354616 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:25.354848 (+ 232us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:25.355644 (+ 796us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:25.355668 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:25.357573 (+ 1905us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:25.360138 (+ 2565us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:25.360146 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:25.360169 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:25.360595 (+ 426us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:25.360973 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:25.360976 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:25.360978 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:25.361197 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:25.361746 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:25.361754 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:25.362138 (+ 384us) client_negotiation.cc:769] Sending connection context
0807 00:04:25.363244 (+ 1106us) client_negotiation.cc:240] Negotiation successful
0807 00:04:25.363681 (+ 437us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":8901,"thread_start_us":172,"threads_started":1}
I20260807 00:04:25.366760 18708 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40641
I20260807 00:04:25.367033 18708 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:25.367805 18708 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:25.369915 18351 ts_manager.cc:194] Registered new tserver with Master: ac8e3304d4ee4e678331b663425cb9cb (127.30.204.65:36561)
I20260807 00:04:25.371541 18351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:40017
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:04:25.401862 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40641
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:25.583205 18723 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:25.583808 18723 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:25.583930 18723 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:25.590431 18723 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:25.590621 18723 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:25.590759 18723 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:04:25.599265 18723 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.30.204.84:46075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18723
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:25.600975 18723 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:25.602428 18723 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:25.617261 18733 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:25.617223 18734 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:25.617611 18723 server_base.cc:1034] running on GCE node
W20260807 00:04:25.623756 18736 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:25.624436 18723 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:25.625247 18723 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:25.626441 18723 hybrid_clock.cc:648] HybridClock initialized: now 1786061065626407 us; error 51 us; skew 500 ppm
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:25.629545 18723 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:04:25.630703 18723 webserver.cc:460] Webserver started at http://127.30.204.66:35393/ using document root <none> and password file <none>
I20260807 00:04:25.631294 18723 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:25.631368 18723 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:25.631633 18723 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:25.648304 18723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "72e5bfb2372f47d6b37374fa9462ca4a"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "755d99d0b5ef8a849c662988a38060d4"
server_key_iv: "fcdf9f07509577c7cfb0f8363af35eeb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:25.649495 18723 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "72e5bfb2372f47d6b37374fa9462ca4a"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "755d99d0b5ef8a849c662988a38060d4"
server_key_iv: "fcdf9f07509577c7cfb0f8363af35eeb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:25.654992 18723 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.000s sys 0.004s
I20260807 00:04:25.671725 18750 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:25.673209 18723 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.002s
I20260807 00:04:25.673367 18723 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "72e5bfb2372f47d6b37374fa9462ca4a"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "755d99d0b5ef8a849c662988a38060d4"
server_key_iv: "fcdf9f07509577c7cfb0f8363af35eeb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:25.673506 18723 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:25.713176 18723 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:25.716339 18723 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:25.716544 18723 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:25.717226 18723 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:25.718191 18723 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:25.718245 18723 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:25.718302 18723 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:25.718330 18723 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:25.729373 18723 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:46613
I20260807 00:04:25.729403 18936 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:46613 every 8 connection(s)
I20260807 00:04:25.730561 18723 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:04:25.732317 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 18723
I20260807 00:04:25.732453 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:04:25.732831 31537 external_mini_cluster.cc:1428] Setting key 5f77b3fa9fc5a0aeb64c03a289aa4afe
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:25.750730 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:25.732847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:38323 (local address 127.30.204.126:40641)
0807 00:04:25.733001 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:25.733006 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:25.735107 (+ 2101us) server_negotiation.cc:407] Connection header received
0807 00:04:25.736439 (+ 1332us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:25.736443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:25.736502 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:25.736555 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:25.738729 (+ 2174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.739512 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.741215 (+ 1703us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.741549 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.741629 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:25.745988 (+ 4359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:25.745993 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:25.745996 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:25.746030 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:25.747832 (+ 1802us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:25.748542 (+ 710us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:25.748547 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:25.748549 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:25.748666 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:25.749207 (+ 541us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:25.749211 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:25.749212 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:25.749471 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:25.749547 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:25.750372 (+ 825us) server_negotiation.cc:299] Negotiation successful
0807 00:04:25.750536 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":64}
I20260807 00:04:25.751916 18940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:25.733392 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40641 (local address 127.30.204.66:38323)
0807 00:04:25.734927 (+ 1535us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:25.734992 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:25.736156 (+ 1164us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:25.737110 (+ 954us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:25.737121 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:25.737726 (+ 605us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:25.738529 (+ 803us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:25.738550 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.739728 (+ 1178us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.739731 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:25.741066 (+ 1335us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:25.741077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:25.741858 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:25.741861 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:25.742074 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:25.743645 (+ 1571us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:25.743669 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:25.745719 (+ 2050us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:25.747967 (+ 2248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:25.747975 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:25.747996 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:25.748400 (+ 404us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:25.748798 (+ 398us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:25.748801 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:25.748804 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:25.749085 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:25.749645 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:25.749652 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:25.750021 (+ 369us) client_negotiation.cc:769] Sending connection context
0807 00:04:25.750311 (+ 290us) client_negotiation.cc:240] Negotiation successful
0807 00:04:25.750719 (+ 408us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1282,"thread_start_us":141,"threads_started":1}
I20260807 00:04:25.753314 18938 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40641
I20260807 00:04:25.753685 18938 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:25.754637 18938 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:25.756049 18351 ts_manager.cc:194] Registered new tserver with Master: 72e5bfb2372f47d6b37374fa9462ca4a (127.30.204.66:46613)
I20260807 00:04:25.756855 18351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:38323
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:04:25.789501 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40641
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:04:25 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061065, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
W20260807 00:04:25.970082 18943 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:25.970402 18943 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:25.970464 18943 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:25.976096 18943 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:25.976192 18943 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:25.976281 18943 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:04:25.984968 18943 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.30.204.84:46075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18943
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:25.986598 18943 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:25.988168 18943 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:26.002919 18955 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:26.004321 18943 server_base.cc:1034] running on GCE node
W20260807 00:04:26.004712 18959 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:26.007714 18954 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:26.008639 18943 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:26.009799 18943 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:26.013145 18943 hybrid_clock.cc:648] HybridClock initialized: now 1786061066013044 us; error 108 us; skew 500 ppm
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:26.017585 18943 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:04:26.018920 18943 webserver.cc:460] Webserver started at http://127.30.204.67:33707/ using document root <none> and password file <none>
I20260807 00:04:26.019879 18943 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:26.019985 18943 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:26.020267 18943 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:26.037524 18943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "0122558b62d244aeaff60316512fc092"
format_stamp: "Formatted at 2026-08-07 00:04:26 on dist-test-slave-pww0"
server_key: "3eb2e096211785791ee9d7369af3c7cd"
server_key_iv: "0f60245c3accee974b5393b32c087448"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:26.038774 18943 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "0122558b62d244aeaff60316512fc092"
format_stamp: "Formatted at 2026-08-07 00:04:26 on dist-test-slave-pww0"
server_key: "3eb2e096211785791ee9d7369af3c7cd"
server_key_iv: "0f60245c3accee974b5393b32c087448"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:26.044456 18943 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.004s sys 0.000s
I20260807 00:04:26.063858 18969 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:26.065414 18943 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.002s sys 0.000s
I20260807 00:04:26.065572 18943 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "0122558b62d244aeaff60316512fc092"
format_stamp: "Formatted at 2026-08-07 00:04:26 on dist-test-slave-pww0"
server_key: "3eb2e096211785791ee9d7369af3c7cd"
server_key_iv: "0f60245c3accee974b5393b32c087448"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:26.065719 18943 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:26.096728 18943 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:26.100409 18943 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:26.100638 18943 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:26.101395 18943 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:26.102578 18943 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:26.102651 18943 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:26.102715 18943 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:26.102743 18943 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:26.113379 18943 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:34443
I20260807 00:04:26.114632 18943 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:04:26.122913 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 18943
I20260807 00:04:26.123052 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:04:26.123368 31537 external_mini_cluster.cc:1428] Setting key 1498cabc0b3daf5334c3fd1cb0d9ede7
I20260807 00:04:26.126827 19155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:34443 every 8 connection(s)
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:26.148039 18719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:26.131311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:39487 (local address 127.30.204.126:40641)
0807 00:04:26.131450 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:26.131474 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:26.132059 (+ 585us) server_negotiation.cc:407] Connection header received
0807 00:04:26.133386 (+ 1327us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:26.133390 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:26.133453 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:26.133505 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:26.135300 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.136153 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.139905 (+ 3752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.140244 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.140324 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:26.143610 (+ 3286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:26.143615 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:26.143618 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:26.143652 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:26.145292 (+ 1640us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:26.146035 (+ 743us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:26.146040 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:26.146043 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:26.146142 (+ 99us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:26.146579 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:26.146582 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:26.146584 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:26.146833 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:26.146909 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:26.147708 (+ 799us) server_negotiation.cc:299] Negotiation successful
0807 00:04:26.147869 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:04:26.152245 19160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:26.129917 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40641 (local address 127.30.204.67:39487)
0807 00:04:26.131858 (+ 1941us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:26.131931 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:26.133093 (+ 1162us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:26.133976 (+ 883us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:26.133986 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:26.134377 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:26.135107 (+ 730us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:26.135128 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.138140 (+ 3012us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.138144 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:26.139753 (+ 1609us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:26.139769 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.140566 (+ 797us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.140570 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:26.140809 (+ 239us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:26.141487 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:26.141512 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:26.143322 (+ 1810us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:26.145475 (+ 2153us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:26.145484 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:26.145507 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:26.145908 (+ 401us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:26.146249 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:26.146253 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:26.146255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:26.146474 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:26.146978 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:26.146985 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:26.147366 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 00:04:26.150514 (+ 3148us) client_negotiation.cc:240] Negotiation successful
0807 00:04:26.151013 (+ 499us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1681,"thread_start_us":235,"threads_started":1}
I20260807 00:04:26.153986 19158 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40641
I20260807 00:04:26.154276 19158 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:26.154964 19158 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:26.156387 18351 ts_manager.cc:194] Registered new tserver with Master: 0122558b62d244aeaff60316512fc092 (127.30.204.67:34443)
I20260807 00:04:26.157162 18351 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:39487
I20260807 00:04:26.162842 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:04:26.163623 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 18477
I20260807 00:04:26.200347 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 18723
I20260807 00:04:26.212064 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 18943
I20260807 00:04:26.228411 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 18304
I20260807 00:04:26.242250 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40641
--webserver_interface=127.30.204.126
--webserver_port=39899
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:40641
--ranger_config_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:26.363547 19161 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:26.363897 19161 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:26.363960 19161 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:26.368702 19161 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:26.368790 19161 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:26.368819 19161 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:26.368840 19161 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:26.368862 19161 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:26.375217 19161 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:40641
--ranger_config_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40641
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=39899
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19161
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:26.376536 19161 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:26.377679 19161 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:04:26.387014 19161 server_base.cc:1034] running on GCE node
W20260807 00:04:26.386962 19173 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:26.386960 19168 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:26.386955 19169 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:26.387895 19161 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:26.389151 19161 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:26.390357 19161 hybrid_clock.cc:648] HybridClock initialized: now 1786061066390334 us; error 39 us; skew 500 ppm
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:26.393388 19161 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:26.394431 19161 webserver.cc:460] Webserver started at http://127.30.204.126:39899/ using document root <none> and password file <none>
I20260807 00:04:26.395020 19161 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:26.395107 19161 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:26.465973 19161 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:04:26.468539 19184 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:26.470129 19161 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 00:04:26.470305 19161 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "4711c8f0a3e34924b81e9092438643fe"
format_stamp: "Formatted at 2026-08-07 00:04:24 on dist-test-slave-pww0"
server_key: "7986ae9b27926ff2a980e161fdb88b80"
server_key_iv: "095f06045da7d3028a903c9a897a6ec5"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:26.470924 19161 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:26.487668 19161 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:26.490939 19161 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:26.491151 19161 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:26.499166 19161 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:40641
I20260807 00:04:26.499204 19275 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:40641 every 8 connection(s)
I20260807 00:04:26.500406 19161 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:26.502041 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 19161
I20260807 00:04:26.502486 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:36561
--local_ip_for_outbound_sockets=127.30.204.65
--tserver_master_addrs=127.30.204.126:40641
--webserver_port=35275
--webserver_interface=127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:04:26.505457 19276 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:04:26.508183 19276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Bootstrap starting.
I20260807 00:04:26.526667 19276 log.cc:824] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:26.534300 19276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:04:26.534754 19276 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Bootstrap complete.
I20260807 00:04:26.538915 19276 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:26.539358 19276 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4711c8f0a3e34924b81e9092438643fe, State: Initialized, Role: FOLLOWER
I20260807 00:04:26.539955 19276 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:26.540251 19276 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:04:26.540349 19276 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:04:26.540458 19276 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:04:26.542737 19276 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:26.543104 19276 leader_election.cc:302] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4711c8f0a3e34924b81e9092438643fe; no voters:
I20260807 00:04:26.543497 19276 leader_election.cc:288] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:04:26.543620 19280 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:04:26.543804 19280 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [term 2 LEADER]: Becoming Leader. State: Replica: 4711c8f0a3e34924b81e9092438643fe, State: Running, Role: LEADER
I20260807 00:04:26.544135 19280 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } }
I20260807 00:04:26.544906 19276 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:26.545680 19282 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "4711c8f0a3e34924b81e9092438643fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } } }
I20260807 00:04:26.545910 19282 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:26.546339 19283 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4711c8f0a3e34924b81e9092438643fe. Latest consensus state: current_term: 2 leader_uuid: "4711c8f0a3e34924b81e9092438643fe" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4711c8f0a3e34924b81e9092438643fe" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40641 } } }
I20260807 00:04:26.546500 19283 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:26.546891 19294 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:26.547331 19276 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260807 00:04:26.553772 19294 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:26.554630 19294 catalog_manager.cc:1259] Loaded cluster ID: e6cbac0486c34e34b2d62839e793299f
I20260807 00:04:26.554746 19294 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:04:26.557725 19294 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:04:26.558787 19294 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Loaded TSK: 0
I20260807 00:04:26.560292 19294 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:04:26.634817 19278 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:26.635121 19278 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:26.635183 19278 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:26.639796 19278 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:26.639873 19278 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:26.639959 19278 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:04:26.646684 19278 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:36561
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=35275
--tserver_master_addrs=127.30.204.126:40641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19278
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:26.648082 19278 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:26.649242 19278 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:26.660758 19336 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:26.661572 19335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:26.661846 19338 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:26.665282 19278 server_base.cc:1034] running on GCE node
I20260807 00:04:26.665922 19278 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:26.666846 19278 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:26.668181 19278 hybrid_clock.cc:648] HybridClock initialized: now 1786061066668127 us; error 79 us; skew 500 ppm
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:26.671896 19278 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:04:26.673189 19278 webserver.cc:460] Webserver started at http://127.30.204.65:35275/ using document root <none> and password file <none>
I20260807 00:04:26.673950 19278 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:26.674067 19278 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:26.708691 19278 fs_manager.cc:709] Time spent opening directory manager: real 0.008s user 0.002s sys 0.001s
I20260807 00:04:26.713408 19349 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:26.714973 19278 fs_manager.cc:725] Time spent opening block manager: real 0.005s user 0.003s sys 0.001s
I20260807 00:04:26.715142 19278 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "ac8e3304d4ee4e678331b663425cb9cb"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "dd49715a0195b8b809c38642c6ca0882"
server_key_iv: "bd6f96f91205763d3a653d999f6994fb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:26.715919 19278 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:26.746526 19278 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:26.749987 19278 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:26.750228 19278 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:26.751009 19278 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:26.752362 19278 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:26.752429 19278 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:26.752488 19278 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:26.752514 19278 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:26.768640 19278 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:36561
I20260807 00:04:26.770202 19278 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:04:26.773645 19541 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:36561 every 8 connection(s)
I20260807 00:04:26.779623 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 19278
I20260807 00:04:26.780182 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:46613
--local_ip_for_outbound_sockets=127.30.204.66
--tserver_master_addrs=127.30.204.126:40641
--webserver_port=35393
--webserver_interface=127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:26.800935 19546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:26.776975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40641 (local address 127.30.204.65:41437)
0807 00:04:26.778007 (+ 1032us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:26.778068 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:26.779208 (+ 1140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:26.781459 (+ 2251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:26.781469 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:26.781865 (+ 396us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:26.782578 (+ 713us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:26.782600 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.784660 (+ 2060us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.784664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:26.786107 (+ 1443us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:26.786123 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.787793 (+ 1670us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.787797 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:26.788010 (+ 213us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:26.792282 (+ 4272us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:26.792304 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:26.793990 (+ 1686us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:26.796892 (+ 2902us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:26.796900 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:26.796919 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:26.797346 (+ 427us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:26.797830 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:26.797834 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:26.797836 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:26.798042 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:26.798986 (+ 944us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:26.798993 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:26.799251 (+ 258us) client_negotiation.cc:769] Sending connection context
0807 00:04:26.799430 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 00:04:26.799851 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":796,"thread_start_us":149,"threads_started":1}
I20260807 00:04:26.800921 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:26.777005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:41437 (local address 127.30.204.126:40641)
0807 00:04:26.777661 (+ 656us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:26.777686 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:26.778232 (+ 546us) server_negotiation.cc:407] Connection header received
0807 00:04:26.779851 (+ 1619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:26.779870 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:26.780364 (+ 494us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:26.780771 (+ 407us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:26.782830 (+ 2059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.784427 (+ 1597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.786289 (+ 1862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:26.787298 (+ 1009us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:26.787476 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:26.794277 (+ 6801us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:26.794285 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:26.794308 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:26.794351 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:26.796694 (+ 2343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:26.797516 (+ 822us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:26.797523 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:26.797530 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:26.797701 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:26.798177 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:26.798181 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:26.798183 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:26.798773 (+ 590us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:26.798908 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:26.799601 (+ 693us) server_negotiation.cc:299] Negotiation successful
0807 00:04:26.799860 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":469,"thread_start_us":120,"threads_started":1}
I20260807 00:04:26.804543 19542 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40641
I20260807 00:04:26.804884 19542 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:26.805526 19542 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
W20260807 00:04:26.808183 19542 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:40641 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 00:04:26.808337 19542 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:26.808559 19542 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:26.809674 19542 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:26.809888 19542 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
W20260807 00:04:26.810844 19542 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:04:26.933679 19548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:26.934047 19548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:26.934098 19548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:26.940058 19548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:26.940163 19548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:26.940253 19548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:04:26.947647 19548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:46613
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=35393
--tserver_master_addrs=127.30.204.126:40641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:26.948983 19548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:26.950191 19548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:26.960937 19557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:26.961118 19561 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:26.961361 19548 server_base.cc:1034] running on GCE node
W20260807 00:04:26.960937 19555 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:26.961891 19548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:26.962674 19548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:26.963889 19548 hybrid_clock.cc:648] HybridClock initialized: now 1786061066963853 us; error 47 us; skew 500 ppm
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:26.967177 19548 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:04:26.968456 19548 webserver.cc:460] Webserver started at http://127.30.204.66:35393/ using document root <none> and password file <none>
I20260807 00:04:26.969107 19548 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:26.969229 19548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:04:26 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:26.997979 19548 fs_manager.cc:709] Time spent opening directory manager: real 0.007s user 0.000s sys 0.003s
I20260807 00:04:27.001837 19572 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:27.003430 19548 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.001s sys 0.000s
I20260807 00:04:27.003623 19548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "72e5bfb2372f47d6b37374fa9462ca4a"
format_stamp: "Formatted at 2026-08-07 00:04:25 on dist-test-slave-pww0"
server_key: "755d99d0b5ef8a849c662988a38060d4"
server_key_iv: "fcdf9f07509577c7cfb0f8363af35eeb"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:27.004232 19548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:27.022811 19548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:27.027027 19548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:27.027254 19548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:27.028031 19548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:27.028971 19548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:27.029021 19548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:27.029078 19548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:27.029107 19548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:27.040457 19548 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:46613
I20260807 00:04:27.041608 19548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:04:27.042464 19758 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:46613 every 8 connection(s)
I20260807 00:04:27.050689 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 19548
I20260807 00:04:27.051213 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.30.204.84:46075/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:34443
--local_ip_for_outbound_sockets=127.30.204.67
--tserver_master_addrs=127.30.204.126:40641
--webserver_port=33707
--webserver_interface=127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41075
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:04:27 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061066, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:27.065676 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:27.043886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:45499 (local address 127.30.204.126:40641)
0807 00:04:27.044082 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:27.044087 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:27.045139 (+ 1052us) server_negotiation.cc:407] Connection header received
0807 00:04:27.046537 (+ 1398us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:27.046539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:27.046611 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:27.046700 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:27.048031 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.048555 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.049736 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.049967 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.050061 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:27.060376 (+ 10315us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:27.060381 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:27.060385 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:27.060428 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:27.062640 (+ 2212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:27.063613 (+ 973us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:27.063618 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:27.063620 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:27.063750 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:27.064297 (+ 547us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:27.064301 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:27.064303 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:27.064576 (+ 273us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:27.064666 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:27.065316 (+ 650us) server_negotiation.cc:299] Negotiation successful
0807 00:04:27.065472 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:04:27.066563 19765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:27.044408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40641 (local address 127.30.204.66:45499)
0807 00:04:27.044958 (+ 550us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:27.045014 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:27.046035 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:27.046957 (+ 922us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:27.046968 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:27.047290 (+ 322us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:27.047876 (+ 586us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:27.047891 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.048692 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.048695 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:27.049606 (+ 911us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:27.049618 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.050132 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.050137 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:27.050317 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:27.054076 (+ 3759us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:27.054162 (+ 86us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:27.059996 (+ 5834us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:27.062863 (+ 2867us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:27.062873 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:27.062897 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:27.063407 (+ 510us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:27.063891 (+ 484us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:27.063894 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:27.063897 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:27.064161 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:27.064735 (+ 574us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:27.064742 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:27.065028 (+ 286us) client_negotiation.cc:769] Sending connection context
0807 00:04:27.065219 (+ 191us) client_negotiation.cc:240] Negotiation successful
0807 00:04:27.065595 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":169,"threads_started":1}
I20260807 00:04:27.068015 19759 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40641
I20260807 00:04:27.068313 19759 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:27.068985 19759 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
W20260807 00:04:27.071125 19759 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:40641 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 00:04:27.071281 19759 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:27.071532 19759 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:27.072688 19759 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:27.072911 19759 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
W20260807 00:04:27.073896 19759 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:04:27.229252 19766 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:27.229660 19766 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:27.229740 19766 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:27.236053 19766 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:27.236152 19766 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:27.236244 19766 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:04:27.244879 19766 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41075
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.30.204.84:46075/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:34443
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=33707
--tserver_master_addrs=127.30.204.126:40641
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19766
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:27.246563 19766 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:27.248185 19766 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:27.262665 19773 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:27.263188 19772 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:27.263636 19776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:27.265345 19766 server_base.cc:1034] running on GCE node
I20260807 00:04:27.265895 19766 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:27.266676 19766 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:27.267890 19766 hybrid_clock.cc:648] HybridClock initialized: now 1786061067267854 us; error 51 us; skew 500 ppm
Aug 07 00:04:27 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:27.270992 19766 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:04:27.272317 19766 webserver.cc:460] Webserver started at http://127.30.204.67:33707/ using document root <none> and password file <none>
I20260807 00:04:27.272996 19766 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:27.273087 19766 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Aug 07 00:04:27 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:27.292832 19766 fs_manager.cc:709] Time spent opening directory manager: real 0.006s user 0.000s sys 0.003s
I20260807 00:04:27.296104 19788 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:27.297803 19766 fs_manager.cc:725] Time spent opening block manager: real 0.004s user 0.002s sys 0.001s
I20260807 00:04:27.297960 19766 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "0122558b62d244aeaff60316512fc092"
format_stamp: "Formatted at 2026-08-07 00:04:26 on dist-test-slave-pww0"
server_key: "3eb2e096211785791ee9d7369af3c7cd"
server_key_iv: "0f60245c3accee974b5393b32c087448"
server_key_version: "kuduclusterkey@0"
I20260807 00:04:27.298731 19766 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:27.314481 19766 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:27.318106 19766 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:27.318334 19766 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:27.319033 19766 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:27.320122 19766 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:27.320187 19766 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:27.320247 19766 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:27.320269 19766 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:27.333808 19766 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:34443
I20260807 00:04:27.335322 19766 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:04:27.336265 19957 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:34443 every 8 connection(s)
I20260807 00:04:27.338896 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 19766
Aug 07 00:04:27 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:27.359016 19964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:27.339941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46144 (local address 127.30.204.126:40641)
0807 00:04:27.340758 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:27.340764 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:27.340780 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:04:27.340828 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:27.340831 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:27.340886 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:27.340980 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:27.342141 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.342917 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.344245 (+ 1328us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.344597 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.344725 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:27.346682 (+ 1957us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:27.346687 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:27.346689 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:27.346722 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:27.351071 (+ 4349us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:27.357419 (+ 6348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:27.357423 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:27.357425 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:27.357529 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:27.357927 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:27.357930 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:27.357932 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:27.358203 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:27.358468 (+ 265us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:27.358655 (+ 187us) server_negotiation.cc:299] Negotiation successful
0807 00:04:27.358809 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":726,"thread_start_us":120,"threads_started":1}
I20260807 00:04:27.361697 19545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:27.337780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:58063 (local address 127.30.204.126:40641)
0807 00:04:27.337936 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:27.337941 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:27.342888 (+ 4947us) server_negotiation.cc:407] Connection header received
0807 00:04:27.344010 (+ 1122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:27.344013 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:27.344071 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:27.344143 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:27.347137 (+ 2994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.347968 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.351243 (+ 3275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.351623 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.351712 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:27.355263 (+ 3551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:27.355268 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:27.355271 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:27.355308 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:27.357271 (+ 1963us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:27.359666 (+ 2395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:27.359669 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:27.359671 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:27.359771 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:27.360296 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:27.360300 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:27.360302 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:27.360540 (+ 238us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:27.360633 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:27.361444 (+ 811us) server_negotiation.cc:299] Negotiation successful
0807 00:04:27.361569 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 00:04:27.363032 19962 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:27.338343 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40641 (local address 127.30.204.67:58063)
0807 00:04:27.342669 (+ 4326us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:27.342756 (+ 87us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:27.343777 (+ 1021us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:27.345204 (+ 1427us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:27.345216 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:27.345647 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:27.346451 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:27.346477 (+ 26us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.348579 (+ 2102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.348583 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:27.350129 (+ 1546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:27.350145 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:27.351964 (+ 1819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:27.351968 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:27.352214 (+ 246us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:27.352966 (+ 752us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:27.352991 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:27.354980 (+ 1989us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:27.357685 (+ 2705us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:27.357694 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:27.357717 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:27.359514 (+ 1797us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:27.359901 (+ 387us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:27.359904 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:27.359906 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:27.360165 (+ 259us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:27.360725 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:27.360733 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:27.361140 (+ 407us) client_negotiation.cc:769] Sending connection context
0807 00:04:27.361397 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 00:04:27.361818 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4030,"thread_start_us":199,"threads_started":1}
I20260807 00:04:27.364408 19958 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40641
I20260807 00:04:27.364701 19958 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:27.365293 19958 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
W20260807 00:04:27.367149 19958 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:40641 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260807 00:04:27.367270 19958 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:27.367497 19958 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:27.368438 19958 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:27.368618 19958 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
W20260807 00:04:27.369445 19958 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Aug 07 00:04:27 dist-test-slave-pww0 krb5kdc[17589](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1786061067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:27.811100 19542 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:27.811518 19542 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:27.812851 19218 ts_manager.cc:194] Registered new tserver with Master: ac8e3304d4ee4e678331b663425cb9cb (127.30.204.65:36561)
I20260807 00:04:27.814314 19218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:41437
I20260807 00:04:28.074129 19759 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:28.074437 19759 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:28.075245 19218 ts_manager.cc:194] Registered new tserver with Master: 72e5bfb2372f47d6b37374fa9462ca4a (127.30.204.66:46613)
I20260807 00:04:28.076071 19218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:45499
Aug 07 00:04:28 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
Aug 07 00:04:28 dist-test-slave-pww0 krb5kdc[17589](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061067, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for HTTP/127.30.204.84@KRBTEST.COM
I20260807 00:04:28.369688 19958 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:28.370046 19958 heartbeater.cc:511] Master 127.30.204.126:40641 requested a full tablet report, sending...
I20260807 00:04:28.374321 19218 ts_manager.cc:194] Registered new tserver with Master: 0122558b62d244aeaff60316512fc092 (127.30.204.67:34443)
I20260807 00:04:28.375041 19218 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:58063
I20260807 00:04:28.385056 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:04:28.397867 19999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.387072 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46154 (local address 127.30.204.126:40641)
0807 00:04:28.387814 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:28.387819 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:28.387832 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:04:28.387875 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:28.387878 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:28.387927 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:28.388018 (+ 91us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:28.388943 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.389811 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.391197 (+ 1386us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.391629 (+ 432us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.391761 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.393309 (+ 1548us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:28.393313 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:28.393316 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:28.393353 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:28.395235 (+ 1882us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:28.395851 (+ 616us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:28.395855 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:28.395857 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:28.395965 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:28.396273 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:28.396277 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:28.396279 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:28.396501 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:28.396624 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:28.397504 (+ 880us) server_negotiation.cc:299] Negotiation successful
0807 00:04:28.397659 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":660,"thread_start_us":134,"threads_started":1}
I20260807 00:04:28.402712 19218 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46154:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:04:28.407742 19218 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:04:28.436874 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.426648 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55550 (local address 127.30.204.67:34443)
0807 00:04:28.429492 (+ 2844us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:28.429501 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:28.429528 (+ 27us) server_negotiation.cc:407] Connection header received
0807 00:04:28.429619 (+ 91us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:28.429625 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:28.429902 (+ 277us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:28.430057 (+ 155us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:28.430884 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.432097 (+ 1213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.433927 (+ 1830us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.434808 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.434888 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.435068 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:28.436390 (+ 1322us) server_negotiation.cc:299] Negotiation successful
0807 00:04:28.436680 (+ 290us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2715,"thread_start_us":107,"threads_started":1}
I20260807 00:04:28.437448 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.426293 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:34443 (local address 127.0.0.1:55550)
0807 00:04:28.428917 (+ 2624us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:28.428987 (+ 70us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:28.429134 (+ 147us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:28.430302 (+ 1168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:28.430310 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:28.430342 (+ 32us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:28.430505 (+ 163us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.430515 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.432242 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.432246 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.433624 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.433639 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.435786 (+ 2147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.435790 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.436110 (+ 320us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.436179 (+ 69us) client_negotiation.cc:769] Sending connection context
0807 00:04:28.437124 (+ 945us) client_negotiation.cc:240] Negotiation successful
0807 00:04:28.437289 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":2385,"thread_start_us":147,"threads_started":1}
I20260807 00:04:28.441370 20009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.427794 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36561 (local address 127.0.0.1:47532)
0807 00:04:28.434172 (+ 6378us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:28.434189 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:28.434303 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:28.435540 (+ 1237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:28.435544 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:28.435570 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:28.435700 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.435708 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.438277 (+ 2569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.438280 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.439523 (+ 1243us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.439541 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.440935 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.440939 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.441028 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.441053 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:04:28.441127 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 00:04:28.441210 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6291,"thread_start_us":129,"threads_started":1}
I20260807 00:04:28.441895 20015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.428637 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:46613 (local address 127.0.0.1:41008)
0807 00:04:28.434730 (+ 6093us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:28.434745 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:28.434852 (+ 107us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:28.436448 (+ 1596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:28.436451 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:28.436477 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:28.436575 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.436581 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.438699 (+ 2118us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.438703 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.440100 (+ 1397us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.440114 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.441507 (+ 1393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.441511 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.441603 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.441630 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:04:28.441694 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 00:04:28.441766 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6027,"thread_start_us":95,"threads_started":1}
I20260807 00:04:28.441584 19856 tablet_service.cc:1467] Processing CreateTablet for tablet 95cfc5f69f394dfab228d759595734ad (DEFAULT_TABLE table=test-table [id=23684d6e9398426c9df22cea4e93bda1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:28.442373 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.428817 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41008 (local address 127.30.204.66:46613)
0807 00:04:28.434573 (+ 5756us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:28.434581 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:28.434956 (+ 375us) server_negotiation.cc:407] Connection header received
0807 00:04:28.435054 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:28.435061 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:28.435340 (+ 279us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:28.435550 (+ 210us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:28.436820 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.437941 (+ 1121us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.440385 (+ 2444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.441170 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.441292 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.441437 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:28.442107 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 00:04:28.442234 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5632,"thread_start_us":111,"threads_started":1}
I20260807 00:04:28.442586 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.428125 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47532 (local address 127.30.204.65:36561)
0807 00:04:28.430807 (+ 2682us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:28.430814 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:28.435127 (+ 4313us) server_negotiation.cc:407] Connection header received
0807 00:04:28.435207 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:28.435212 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:28.435359 (+ 147us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:28.435550 (+ 191us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:28.435859 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.438136 (+ 2277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.439709 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.440573 (+ 864us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.440699 (+ 126us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.440865 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:28.442243 (+ 1378us) server_negotiation.cc:299] Negotiation successful
0807 00:04:28.442360 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2552,"thread_start_us":113,"threads_started":1}
I20260807 00:04:28.442958 19856 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95cfc5f69f394dfab228d759595734ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:28.447538 19620 tablet_service.cc:1467] Processing CreateTablet for tablet 95cfc5f69f394dfab228d759595734ad (DEFAULT_TABLE table=test-table [id=23684d6e9398426c9df22cea4e93bda1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:28.448779 19620 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95cfc5f69f394dfab228d759595734ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:28.450476 19424 tablet_service.cc:1467] Processing CreateTablet for tablet 95cfc5f69f394dfab228d759595734ad (DEFAULT_TABLE table=test-table [id=23684d6e9398426c9df22cea4e93bda1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:28.452742 19424 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 95cfc5f69f394dfab228d759595734ad. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:28.459105 20018 tablet_bootstrap.cc:492] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: Bootstrap starting.
I20260807 00:04:28.461894 20018 tablet_bootstrap.cc:654] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:28.462924 20018 log.cc:824] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:28.464274 20019 tablet_bootstrap.cc:492] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: Bootstrap starting.
I20260807 00:04:28.466653 20019 tablet_bootstrap.cc:654] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:28.467694 20019 log.cc:824] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:28.468629 20020 tablet_bootstrap.cc:492] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb: Bootstrap starting.
I20260807 00:04:28.470481 20018 tablet_bootstrap.cc:492] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: No bootstrap required, opened a new log
I20260807 00:04:28.470742 20018 ts_tablet_manager.cc:1397] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: Time spent bootstrapping tablet: real 0.012s user 0.007s sys 0.000s
I20260807 00:04:28.474656 20020 tablet_bootstrap.cc:654] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:28.475075 20018 raft_consensus.cc:348] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.475509 20018 raft_consensus.cc:374] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:28.475584 20018 raft_consensus.cc:729] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0122558b62d244aeaff60316512fc092, State: Initialized, Role: FOLLOWER
I20260807 00:04:28.476184 20020 log.cc:824] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:28.476174 20018 consensus_queue.cc:260] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.477495 20018 ts_tablet_manager.cc:1428] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: Time spent starting tablet: real 0.007s user 0.002s sys 0.005s
I20260807 00:04:28.477923 19958 heartbeater.cc:503] Master 127.30.204.126:40641 was elected leader, sending a full tablet report...
I20260807 00:04:28.480585 20019 tablet_bootstrap.cc:492] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: No bootstrap required, opened a new log
I20260807 00:04:28.480862 20019 ts_tablet_manager.cc:1397] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: Time spent bootstrapping tablet: real 0.017s user 0.003s sys 0.002s
I20260807 00:04:28.485491 20019 raft_consensus.cc:348] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.485929 20019 raft_consensus.cc:374] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:28.486002 20019 raft_consensus.cc:729] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 72e5bfb2372f47d6b37374fa9462ca4a, State: Initialized, Role: FOLLOWER
I20260807 00:04:28.486642 20019 consensus_queue.cc:260] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.488119 20019 ts_tablet_manager.cc:1428] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: Time spent starting tablet: real 0.007s user 0.006s sys 0.001s
I20260807 00:04:28.488535 19759 heartbeater.cc:503] Master 127.30.204.126:40641 was elected leader, sending a full tablet report...
I20260807 00:04:28.489624 20020 tablet_bootstrap.cc:492] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb: No bootstrap required, opened a new log
I20260807 00:04:28.490023 20020 ts_tablet_manager.cc:1397] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb: Time spent bootstrapping tablet: real 0.022s user 0.004s sys 0.004s
I20260807 00:04:28.494164 20020 raft_consensus.cc:348] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.494445 20020 raft_consensus.cc:374] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:28.494484 20020 raft_consensus.cc:729] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ac8e3304d4ee4e678331b663425cb9cb, State: Initialized, Role: FOLLOWER
I20260807 00:04:28.496260 20020 consensus_queue.cc:260] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.497812 20020 ts_tablet_manager.cc:1428] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb: Time spent starting tablet: real 0.008s user 0.005s sys 0.000s
I20260807 00:04:28.498219 19542 heartbeater.cc:503] Master 127.30.204.126:40641 was elected leader, sending a full tablet report...
W20260807 00:04:28.527626 19544 tablet.cc:2369] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:04:28.544381 19760 tablet.cc:2369] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:04:28.587853 19961 tablet.cc:2369] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:04:28.592525 20031 raft_consensus.cc:482] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:28.592770 20031 raft_consensus.cc:504] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.594159 20031 leader_election.cc:288] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0122558b62d244aeaff60316512fc092 (127.30.204.67:34443), 72e5bfb2372f47d6b37374fa9462ca4a (127.30.204.66:46613)
I20260807 00:04:28.600140 20016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.594793 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:55669 (local address 127.30.204.66:46613)
0807 00:04:28.594932 (+ 139us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:28.594937 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:28.595567 (+ 630us) server_negotiation.cc:407] Connection header received
0807 00:04:28.595801 (+ 234us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:28.595805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:28.595857 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:28.595906 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:28.596406 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.597156 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.598734 (+ 1578us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.599373 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.599502 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.599558 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:28.599888 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 00:04:28.599979 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:04:28.600670 20032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.594482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:46613 (local address 127.30.204.65:55669)
0807 00:04:28.595428 (+ 946us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:28.595444 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:28.595665 (+ 221us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:28.596054 (+ 389us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:28.596057 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:28.596076 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:28.596238 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.596246 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.597286 (+ 1040us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.597289 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.598603 (+ 1314us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.598611 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.599656 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.599659 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.599752 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.599777 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:04:28.600399 (+ 622us) client_negotiation.cc:240] Negotiation successful
0807 00:04:28.600484 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":862,"mutex_wait_us":565,"thread_start_us":126,"threads_started":1}
I20260807 00:04:28.601372 20011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.594931 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:42707 (local address 127.30.204.67:34443)
0807 00:04:28.595039 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:28.595042 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:28.596865 (+ 1823us) server_negotiation.cc:407] Connection header received
0807 00:04:28.597046 (+ 181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:28.597049 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:28.597096 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:28.597155 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:28.597539 (+ 384us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.598319 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.599773 (+ 1454us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.600317 (+ 544us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.600418 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.600466 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:28.601167 (+ 701us) server_negotiation.cc:299] Negotiation successful
0807 00:04:28.601258 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:04:28.601420 19680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cfc5f69f394dfab228d759595734ad" candidate_uuid: "ac8e3304d4ee4e678331b663425cb9cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" is_pre_election: true
I20260807 00:04:28.601786 19680 raft_consensus.cc:2454] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac8e3304d4ee4e678331b663425cb9cb in term 0.
I20260807 00:04:28.602093 20033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.594482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:34443 (local address 127.30.204.65:42707)
0807 00:04:28.596761 (+ 2279us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:28.596778 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:28.596938 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:28.597286 (+ 348us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:28.597289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:28.597305 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:28.597417 (+ 112us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.597423 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.598493 (+ 1070us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.598496 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.599638 (+ 1142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:28.599645 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.600916 (+ 1271us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.600919 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:28.601015 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.601037 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:04:28.601784 (+ 747us) client_negotiation.cc:240] Negotiation successful
0807 00:04:28.601951 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":826,"thread_start_us":125,"threads_started":1}
I20260807 00:04:28.602384 19358 leader_election.cc:302] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72e5bfb2372f47d6b37374fa9462ca4a, ac8e3304d4ee4e678331b663425cb9cb; no voters:
I20260807 00:04:28.602752 20031 raft_consensus.cc:2781] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:04:28.602841 20031 raft_consensus.cc:482] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:04:28.602881 20031 raft_consensus.cc:3037] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:28.602932 19884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cfc5f69f394dfab228d759595734ad" candidate_uuid: "ac8e3304d4ee4e678331b663425cb9cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0122558b62d244aeaff60316512fc092" is_pre_election: true
I20260807 00:04:28.603251 19884 raft_consensus.cc:2454] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ac8e3304d4ee4e678331b663425cb9cb in term 0.
I20260807 00:04:28.604323 20031 raft_consensus.cc:504] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.604707 20031 leader_election.cc:288] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [CANDIDATE]: Term 1 election: Requested vote from peers 0122558b62d244aeaff60316512fc092 (127.30.204.67:34443), 72e5bfb2372f47d6b37374fa9462ca4a (127.30.204.66:46613)
I20260807 00:04:28.605142 19884 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cfc5f69f394dfab228d759595734ad" candidate_uuid: "ac8e3304d4ee4e678331b663425cb9cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0122558b62d244aeaff60316512fc092"
I20260807 00:04:28.605142 19680 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "95cfc5f69f394dfab228d759595734ad" candidate_uuid: "ac8e3304d4ee4e678331b663425cb9cb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "72e5bfb2372f47d6b37374fa9462ca4a"
I20260807 00:04:28.605278 19680 raft_consensus.cc:3037] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:28.605278 19884 raft_consensus.cc:3037] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:28.606643 19680 raft_consensus.cc:2454] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac8e3304d4ee4e678331b663425cb9cb in term 1.
I20260807 00:04:28.606659 19884 raft_consensus.cc:2454] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ac8e3304d4ee4e678331b663425cb9cb in term 1.
I20260807 00:04:28.607048 19358 leader_election.cc:302] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 72e5bfb2372f47d6b37374fa9462ca4a, ac8e3304d4ee4e678331b663425cb9cb; no voters:
I20260807 00:04:28.607316 20031 raft_consensus.cc:2781] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:28.607599 20031 raft_consensus.cc:686] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [term 1 LEADER]: Becoming Leader. State: Replica: ac8e3304d4ee4e678331b663425cb9cb, State: Running, Role: LEADER
I20260807 00:04:28.608031 20031 consensus_queue.cc:237] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } }
I20260807 00:04:28.612270 19218 catalog_manager.cc:5495] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb reported cstate change: term changed from 0 to 1, leader changed from <none> to ac8e3304d4ee4e678331b663425cb9cb (127.30.204.65). New cstate: current_term: 1 leader_uuid: "ac8e3304d4ee4e678331b663425cb9cb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ac8e3304d4ee4e678331b663425cb9cb" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36561 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 } health_report { overall_health: UNKNOWN } } }
I20260807 00:04:28.636437 20014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.630916 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47544 (local address 127.30.204.65:36561)
0807 00:04:28.631042 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:28.631046 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:28.631061 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:04:28.631170 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:28.631173 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:28.631246 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:28.631370 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:04:28.631820 (+ 450us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.632632 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.633929 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:28.634268 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:28.634374 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:28.634857 (+ 483us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:04:28.635528 (+ 671us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:04:28.635652 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:28.636176 (+ 524us) server_negotiation.cc:299] Negotiation successful
0807 00:04:28.636252 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260807 00:04:28.644860 19680 raft_consensus.cc:1261] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 1 FOLLOWER]: Refusing update from remote peer ac8e3304d4ee4e678331b663425cb9cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:28.645895 20039 consensus_queue.cc:1035] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "72e5bfb2372f47d6b37374fa9462ca4a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46613 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:28.658845 19884 raft_consensus.cc:1261] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 1 FOLLOWER]: Refusing update from remote peer ac8e3304d4ee4e678331b663425cb9cb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:28.659647 20039 consensus_queue.cc:1035] T 95cfc5f69f394dfab228d759595734ad P ac8e3304d4ee4e678331b663425cb9cb [LEADER]: Connected to new peer: Peer: permanent_uuid: "0122558b62d244aeaff60316512fc092" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34443 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:28.670485 19218 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:28.670701 19218 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:46154:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:28.670944 20044 mvcc.cc:205] Tried to move back new op lower bound from 7315706137159610368 to 7315706137022500864. Current Snapshot: MvccSnapshot[applied={T|T < 7315706137159610368 or (T in {7315706137159610368})}]
I20260807 00:04:28.674067 20045 mvcc.cc:205] Tried to move back new op lower bound from 7315706137159610368 to 7315706137022500864. Current Snapshot: MvccSnapshot[applied={T|T < 7315706137159610368}]
I20260807 00:04:28.674787 19218 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4711c8f0a3e34924b81e9092438643fe: Sending DeleteTablet for 3 replicas of tablet 95cfc5f69f394dfab228d759595734ad
I20260807 00:04:28.676010 19424 tablet_service.cc:1514] Processing DeleteTablet for tablet 95cfc5f69f394dfab228d759595734ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:28 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:47532
I20260807 00:04:28.676153 19856 tablet_service.cc:1514] Processing DeleteTablet for tablet 95cfc5f69f394dfab228d759595734ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:28 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:55550
I20260807 00:04:28.676685 19620 tablet_service.cc:1514] Processing DeleteTablet for tablet 95cfc5f69f394dfab228d759595734ad with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:28 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:41008
I20260807 00:04:28.677389 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 19278
I20260807 00:04:28.687738 20056 tablet_replica.cc:330] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092: stopping tablet replica
I20260807 00:04:28.688251 20056 raft_consensus.cc:2229] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:04:28.688651 20056 raft_consensus.cc:2258] T 95cfc5f69f394dfab228d759595734ad P 0122558b62d244aeaff60316512fc092 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:28.702001 20057 tablet_replica.cc:330] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: stopping tablet replica
I20260807 00:04:28.702469 20057 raft_consensus.cc:2229] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:04:28.702925 20057 raft_consensus.cc:2258] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:28.709610 20057 ts_tablet_manager.cc:1905] T 95cfc5f69f394dfab228d759595734ad P 72e5bfb2372f47d6b37374fa9462ca4a: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 00:04:28.710033 19791 connection.cc:535] server connection from 127.30.204.65:42707 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
W20260807 00:04:28.711108 19186 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:36561 (error 108)
I20260807 00:04:28.712141 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 19548
I20260807 00:04:28.712909 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.712511 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36561 (local address 127.0.0.1:47556)
0807 00:04:28.712718 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:28.712801 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:36561: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
W20260807 00:04:28.714788 19186 catalog_manager.cc:4584] TS ac8e3304d4ee4e678331b663425cb9cb (127.30.204.65:36561): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 95cfc5f69f394dfab228d759595734ad: Network error: Client connection negotiation failed: client connection to 127.30.204.65:36561: connect: Connection refused (error 111)
I20260807 00:04:28.725244 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 19766
I20260807 00:04:28.741743 20012 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:28.741225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:46613 (local address 127.0.0.1:41018)
0807 00:04:28.741527 (+ 302us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:28.741621 (+ 94us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.66:46613: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199}
I20260807 00:04:28.752408 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 19161
I20260807 00:04:28.774644 31537 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260807 00:04:29.120875 31537 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260807 00:04:29.121009 31537 mini_ranger.cc:66] Stopping Ranger...
I20260807 00:04:29.466955 31537 mini_ranger.cc:68] Stopped Ranger
2026-08-07 00:04:29.467 UTC [17649] LOG: received smart shutdown request
2026-08-07 00:04:29.472 UTC [17649] LOG: background worker "logical replication launcher" (PID 17663) exited with exit code 1
2026-08-07 00:04:29.472 UTC [17655] LOG: shutting down
2026-08-07 00:04:29.661 UTC [17649] LOG: database system is shut down
2026-08-07T00:04:29Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51069 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:04:29 dist-test-slave-pww0 krb5kdc[20082](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:04:29 dist-test-slave-pww0 krb5kdc[20082](info): set up 2 sockets
Aug 07 00:04:29 dist-test-slave-pww0 krb5kdc[20082](info): commencing operation
krb5kdc: starting...
W20260807 00:04:30.937412 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.148s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:04:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:04:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:04:31.075502 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:42947
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:42947,127.30.204.125:42135,127.30.204.124:41987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:31.199584 20101 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:31.199918 20101 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:31.199990 20101 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:31.204226 20101 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:31.204319 20101 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:31.204350 20101 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:31.204380 20101 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:31.204416 20101 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:31.210259 20101 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:42947,127.30.204.125:42135,127.30.204.124:41987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:42947
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20101
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:31.211625 20101 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:31.212829 20101 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:31.222147 20114 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:31.222204 20101 server_base.cc:1034] running on GCE node
W20260807 00:04:31.222141 20111 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:31.222134 20110 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:31.223111 20101 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:31.224398 20101 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:31.225576 20101 hybrid_clock.cc:648] HybridClock initialized: now 1786061071225551 us; error 38 us; skew 500 ppm
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:31.228652 20101 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:31.229718 20101 webserver.cc:460] Webserver started at http://127.30.204.126:41469/ using document root <none> and password file <none>
I20260807 00:04:31.230350 20101 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:31.230430 20101 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:31.230656 20101 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:31.233001 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "aae7ce3994704891897169f287d921f1"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "c6eda4b41c7c522a65b5e8c6ff43f3e3"
server_key_iv: "06262a776c1baa5ec650612470fdd8c1"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.233625 20101 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "aae7ce3994704891897169f287d921f1"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "c6eda4b41c7c522a65b5e8c6ff43f3e3"
server_key_iv: "06262a776c1baa5ec650612470fdd8c1"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.237771 20101 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:04:31.240412 20125 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:31.241720 20101 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 00:04:31.241847 20101 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "aae7ce3994704891897169f287d921f1"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "c6eda4b41c7c522a65b5e8c6ff43f3e3"
server_key_iv: "06262a776c1baa5ec650612470fdd8c1"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.241953 20101 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:31.257324 20101 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:31.261004 20101 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:31.261235 20101 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:31.268801 20101 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:42947
I20260807 00:04:31.269084 20208 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:42947 every 8 connection(s)
I20260807 00:04:31.269914 20101 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:31.273097 20210 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:31.274047 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 20101
I20260807 00:04:31.274209 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:04:31.274539 31537 external_mini_cluster.cc:1428] Setting key ecc78e9e365678004f9fc2ecd569d9c9
I20260807 00:04:31.278053 20210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.204.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.125@KRBTEST.COM" created.
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:31.294385 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.280763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:42947 (local address 127.0.0.1:47220)
0807 00:04:31.281373 (+ 610us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.281414 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.282297 (+ 883us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.283021 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.283027 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.283515 (+ 488us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.283815 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.283832 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.285043 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.285046 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.285884 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.285895 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.286683 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.286686 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.286813 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.287377 (+ 564us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.287395 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.289101 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.291033 (+ 1932us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.291043 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.291057 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.291374 (+ 317us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.291723 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.291726 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.291729 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.291854 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.292488 (+ 634us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.292495 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.292647 (+ 152us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.292808 (+ 161us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.293269 (+ 461us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":437,"mutex_wait_us":113,"spinlock_wait_cycles":23808,"thread_start_us":169,"threads_started":1}
I20260807 00:04:31.294382 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.280762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47220 (local address 127.30.204.126:42947)
0807 00:04:31.281259 (+ 497us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.281279 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.281527 (+ 248us) server_negotiation.cc:407] Connection header received
0807 00:04:31.282616 (+ 1089us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.282620 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.282825 (+ 205us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.283003 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.283971 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.284918 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.286004 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.286494 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.286660 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.289264 (+ 2604us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.289272 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.289284 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.289307 (+ 23us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.290912 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.291501 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.291507 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.291508 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.291624 (+ 116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.291941 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.291945 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.291946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.292374 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.292505 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.292894 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.293204 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":42,"server-negotiator.queue_time_us":337,"thread_start_us":137,"threads_started":1}
I20260807 00:04:31.297164 20210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:04:31.298023 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.297729 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35930)
0807 00:04:31.297860 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.297946 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 00:04:31.298282 20130 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
I20260807 00:04:31.299417 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.299178 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35942)
0807 00:04:31.299295 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.299348 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 00:04:31.299615 20210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:42135: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab.
Entry for principal kudu/127.30.204.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab.
I20260807 00:04:31.303331 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:42135
--webserver_interface=127.30.204.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab
--principal=kudu/127.30.204.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:42947,127.30.204.125:42135,127.30.204.124:41987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:04:31.348912 20210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } attempt: 1
I20260807 00:04:31.349946 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.349572 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35950)
0807 00:04:31.349761 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.349842 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 00:04:31.350811 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.350546 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35956)
0807 00:04:31.350668 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.350735 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 00:04:31.350988 20210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:42135: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
I20260807 00:04:31.419152 20210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } attempt: 2
I20260807 00:04:31.420075 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.419740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35964)
0807 00:04:31.419902 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.419975 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 00:04:31.420922 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.420668 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35978)
0807 00:04:31.420785 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.420853 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
W20260807 00:04:31.421084 20210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:42135: Network error: Client connection negotiation failed: client connection to 127.30.204.125:42135: connect: Connection refused (error 111)
W20260807 00:04:31.423661 20220 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:31.423952 20220 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:31.424026 20220 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:31.428229 20220 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:31.428303 20220 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:31.428331 20220 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:31.428362 20220 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:31.428393 20220 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:31.434039 20220 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:42947,127.30.204.125:42135,127.30.204.124:41987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab
--principal=kudu/127.30.204.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:42135
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.204.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20220
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:31.435264 20220 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:31.436431 20220 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:31.445428 20229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:31.445541 20220 server_base.cc:1034] running on GCE node
W20260807 00:04:31.445664 20233 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:31.445429 20230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:31.446408 20220 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:31.447702 20220 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:31.448884 20220 hybrid_clock.cc:648] HybridClock initialized: now 1786061071448858 us; error 38 us; skew 500 ppm
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:31.451855 20220 init.cc:377] Logged in from keytab as kudu/127.30.204.125@KRBTEST.COM (short username kudu)
I20260807 00:04:31.452860 20220 webserver.cc:460] Webserver started at http://127.30.204.125:34729/ using document root <none> and password file <none>
I20260807 00:04:31.453424 20220 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:31.453487 20220 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:31.453709 20220 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:31.455986 20220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data/instance:
uuid: "e9de06e6410842c8ae0595dcdb0cf649"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "52d26e0eef4339dd023f1ce7cda798b8"
server_key_iv: "a5419be5da11bbf9818acb00a1dc99cf"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.456593 20220 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/wal/instance:
uuid: "e9de06e6410842c8ae0595dcdb0cf649"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "52d26e0eef4339dd023f1ce7cda798b8"
server_key_iv: "a5419be5da11bbf9818acb00a1dc99cf"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.460742 20220 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:04:31.463706 20241 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:31.465417 20220 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:04:31.465574 20220 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/wal
uuid: "e9de06e6410842c8ae0595dcdb0cf649"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "52d26e0eef4339dd023f1ce7cda798b8"
server_key_iv: "a5419be5da11bbf9818acb00a1dc99cf"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.465708 20220 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:31.475574 20220 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:31.478842 20220 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:31.479035 20220 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:31.486176 20220 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.125:42135
I20260807 00:04:31.486447 20313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.125:42135 every 8 connection(s)
I20260807 00:04:31.487208 20220 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
I20260807 00:04:31.490247 20314 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:31.491555 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 20220
I20260807 00:04:31.491698 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-1/wal/instance
I20260807 00:04:31.492050 31537 external_mini_cluster.cc:1428] Setting key 78f84424c56913f7281536cde78db292
I20260807 00:04:31.495330 20314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.204.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.124@KRBTEST.COM" created.
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:31.509716 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.497493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47226 (local address 127.30.204.126:42947)
0807 00:04:31.497672 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.497678 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.498656 (+ 978us) server_negotiation.cc:407] Connection header received
0807 00:04:31.499686 (+ 1030us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.499690 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.499761 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.499871 (+ 110us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.501091 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.501681 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.503020 (+ 1339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.503307 (+ 287us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.503406 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.506193 (+ 2787us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.506196 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.506198 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.506228 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.507542 (+ 1314us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.508069 (+ 527us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.508073 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.508074 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.508176 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.508482 (+ 306us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.508485 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.508487 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.508735 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.508833 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.509366 (+ 533us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.509519 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:04:31.510516 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.497886 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:42947 (local address 127.0.0.1:47226)
0807 00:04:31.498462 (+ 576us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.498522 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.499422 (+ 900us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.500018 (+ 596us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.500026 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.500394 (+ 368us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.500911 (+ 517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.500925 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.501851 (+ 926us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.501854 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.502858 (+ 1004us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.502871 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.503485 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.503490 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.503660 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.504289 (+ 629us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.504307 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.505952 (+ 1645us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.507655 (+ 1703us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.507660 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.507675 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.507954 (+ 279us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.508260 (+ 306us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.508262 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.508264 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.508399 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.508820 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.508825 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.509094 (+ 269us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.509261 (+ 167us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.509583 (+ 322us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":141,"threads_started":1}
I20260807 00:04:31.511941 20314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:04:31.512248 20210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } attempt: 3
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:04:31.522065 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.512455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35986)
0807 00:04:31.512610 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.512623 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.512706 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.513505 (+ 799us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.513509 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.513693 (+ 184us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.513828 (+ 135us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.513835 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.514936 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.514940 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.515510 (+ 570us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.515522 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.516124 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.516127 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.516252 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.516683 (+ 431us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.516699 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.517916 (+ 1217us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.520329 (+ 2413us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.520331 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.520333 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.520584 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.520916 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.520920 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.520922 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.521005 (+ 83us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.521534 (+ 529us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.521538 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.521632 (+ 94us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.521744 (+ 112us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.521888 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":78,"mutex_wait_us":3}
I20260807 00:04:31.522233 20321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.512862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35986 (local address 127.30.204.125:42135)
0807 00:04:31.513159 (+ 297us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.513168 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.513178 (+ 10us) server_negotiation.cc:407] Connection header received
0807 00:04:31.513201 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.513204 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.513349 (+ 145us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.513491 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.513951 (+ 460us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.514768 (+ 817us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.515611 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.516026 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.516122 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.518057 (+ 1935us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.518068 (+ 11us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.518082 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.518104 (+ 22us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.520200 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.520692 (+ 492us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.520699 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.520701 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.520793 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.521093 (+ 300us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.521096 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.521098 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.521431 (+ 333us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.521556 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.521854 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.522012 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":74,"server-negotiator.queue_time_us":246,"spinlock_wait_cycles":7296,"thread_start_us":144,"threads_started":1}
I20260807 00:04:31.522411 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.512661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:35990)
0807 00:04:31.512856 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.512869 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.512954 (+ 85us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.513702 (+ 748us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.513704 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.513852 (+ 148us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.513972 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.513979 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.514910 (+ 931us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.514914 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.515584 (+ 670us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.515595 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.516157 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.516161 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.516279 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.516802 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.516816 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.518468 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.520830 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.520833 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.520836 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.521041 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.521284 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.521287 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.521289 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.521406 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.521783 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.521787 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.521912 (+ 125us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.522050 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.522212 (+ 162us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab.
Entry for principal kudu/127.30.204.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab.
I20260807 00:04:31.523291 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.512676 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35990 (local address 127.30.204.125:42135)
0807 00:04:31.513049 (+ 373us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.513062 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.513084 (+ 22us) server_negotiation.cc:407] Connection header received
0807 00:04:31.513126 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.513131 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.513349 (+ 218us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.513491 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.514085 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.514814 (+ 729us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.515703 (+ 889us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.516051 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.516183 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.518621 (+ 2438us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.518624 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.518626 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.518651 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.520694 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.521132 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.521134 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.521135 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.521202 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.521492 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.521495 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.521496 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.521679 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.521790 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.522119 (+ 329us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.523123 (+ 1004us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":122,"server-negotiator.queue_time_us":276,"spinlock_wait_cycles":8832,"thread_start_us":181,"threads_started":1}
I20260807 00:04:31.523753 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:41987
--webserver_interface=127.30.204.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab
--principal=kudu/127.30.204.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46741
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:42947,127.30.204.125:42135,127.30.204.124:41987
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:04:31.524403 20314 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:04:31.524578 20210 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:04:31.525156 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.524829 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51638)
0807 00:04:31.525000 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.525085 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":93}
I20260807 00:04:31.525311 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.525049 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51642)
0807 00:04:31.525182 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.525240 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
W20260807 00:04:31.525388 20243 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
I20260807 00:04:31.526224 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.526031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51652)
0807 00:04:31.526128 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.526174 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 00:04:31.526278 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.525971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51650)
0807 00:04:31.526149 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.526207 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260807 00:04:31.526396 20210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:41987: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
W20260807 00:04:31.526440 20314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:41987: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
I20260807 00:04:31.557564 20210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } attempt: 1
I20260807 00:04:31.558571 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.558199 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51654)
0807 00:04:31.558396 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.558475 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":83}
I20260807 00:04:31.559300 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.559074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51668)
0807 00:04:31.559180 (+ 106us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.559234 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
W20260807 00:04:31.559425 20210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:41987: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
I20260807 00:04:31.575737 20314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } attempt: 1
I20260807 00:04:31.576761 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.576379 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51678)
0807 00:04:31.576571 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.576668 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 00:04:31.577545 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.577314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51690)
0807 00:04:31.577433 (+ 119us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.577484 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 00:04:31.577765 20314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:41987: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
I20260807 00:04:31.634637 20210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } attempt: 2
I20260807 00:04:31.635656 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.635249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51692)
0807 00:04:31.635438 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.635552 (+ 114us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":70}
I20260807 00:04:31.636520 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.636241 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51700)
0807 00:04:31.636375 (+ 134us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.636435 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 00:04:31.636708 20210 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:41987: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
W20260807 00:04:31.644352 20323 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:31.644646 20323 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:31.644721 20323 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 00:04:31.645925 20314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } attempt: 2
I20260807 00:04:31.646835 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.646495 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51710)
0807 00:04:31.646690 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.646750 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":97}
I20260807 00:04:31.647778 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.647536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:51714)
0807 00:04:31.647654 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.647708 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 00:04:31.647907 20314 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:41987: Network error: Client connection negotiation failed: client connection to 127.30.204.124:41987: connect: Connection refused (error 111)
W20260807 00:04:31.649243 20323 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:31.649304 20323 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:31.649333 20323 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:31.649361 20323 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:31.649384 20323 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:31.655030 20323 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46741
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:42947,127.30.204.125:42135,127.30.204.124:41987
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab
--principal=kudu/127.30.204.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:41987
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.204.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20323
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:31.656277 20323 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:31.657385 20323 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:31.666491 20335 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:31.666484 20339 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:31.666566 20323 server_base.cc:1034] running on GCE node
W20260807 00:04:31.666484 20347 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:31.667543 20323 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:31.668771 20323 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:31.670043 20323 hybrid_clock.cc:648] HybridClock initialized: now 1786061071670008 us; error 49 us; skew 500 ppm
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:31.673033 20323 init.cc:377] Logged in from keytab as kudu/127.30.204.124@KRBTEST.COM (short username kudu)
I20260807 00:04:31.674052 20323 webserver.cc:460] Webserver started at http://127.30.204.124:41695/ using document root <none> and password file <none>
I20260807 00:04:31.674644 20323 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:31.674721 20323 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:31.674938 20323 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:31.677191 20323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data/instance:
uuid: "4a50fc28c13242b899b17afa805c7764"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "d75248d4aa7c08da27616bc7c87dff70"
server_key_iv: "47e67bc0d3c249ab7fe38dba16dc21a3"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.677788 20323 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/wal/instance:
uuid: "4a50fc28c13242b899b17afa805c7764"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "d75248d4aa7c08da27616bc7c87dff70"
server_key_iv: "47e67bc0d3c249ab7fe38dba16dc21a3"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.681869 20323 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:04:31.684424 20355 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:31.685743 20323 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 00:04:31.685868 20323 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/wal
uuid: "4a50fc28c13242b899b17afa805c7764"
format_stamp: "Formatted at 2026-08-07 00:04:31 on dist-test-slave-pww0"
server_key: "d75248d4aa7c08da27616bc7c87dff70"
server_key_iv: "47e67bc0d3c249ab7fe38dba16dc21a3"
server_key_version: "encryptionkey@0"
I20260807 00:04:31.685969 20323 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:31.713069 20323 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:31.716224 20323 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:31.716423 20323 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:31.726024 20323 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.124:41987
I20260807 00:04:31.726215 20446 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.124:41987 every 8 connection(s)
I20260807 00:04:31.727116 20323 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
I20260807 00:04:31.730486 20448 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:31.732494 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 20323
I20260807 00:04:31.732637 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACert.1786060901439409-31537-0/minicluster-data/master-2/wal/instance
I20260807 00:04:31.732945 31537 external_mini_cluster.cc:1428] Setting key fd7862fe805622f00d4b41ede257d55a
I20260807 00:04:31.735867 20210 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } attempt: 3
I20260807 00:04:31.735249 20448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:04:31.739071 20314 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } attempt: 3
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
I20260807 00:04:31.749316 20217 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.736323 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:42616)
0807 00:04:31.736530 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.736545 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.736639 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.737081 (+ 442us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.737084 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.737269 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.737399 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.737407 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.739487 (+ 2080us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.739501 (+ 14us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.740358 (+ 857us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.740369 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.741478 (+ 1109us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.741483 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.741570 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.742118 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.742133 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.743711 (+ 1578us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.747127 (+ 3416us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.747131 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.747133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.747388 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.747988 (+ 600us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.747991 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.747993 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.748092 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.748803 (+ 711us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.748806 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.748905 (+ 99us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.748987 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.749131 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132}
I20260807 00:04:31.749590 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.737803 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46546 (local address 127.30.204.126:42947)
0807 00:04:31.737918 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.737922 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.738284 (+ 362us) server_negotiation.cc:407] Connection header received
0807 00:04:31.738471 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.738474 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.738517 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.738557 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.739450 (+ 893us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.740206 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.741930 (+ 1724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.742228 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.742307 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.745296 (+ 2989us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.745301 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.745303 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.745330 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.747369 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.747917 (+ 548us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.747921 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.747923 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.748004 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.748355 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.748359 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.748361 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.748682 (+ 321us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.748758 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.749318 (+ 560us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.749439 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":55,"server-negotiator.queue_time_us":47}
I20260807 00:04:31.751199 20452 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.734548 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42606 (local address 127.30.204.124:41987)
0807 00:04:31.735028 (+ 480us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.735045 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.735084 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:04:31.736098 (+ 1014us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.736130 (+ 32us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.736586 (+ 456us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.736971 (+ 385us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.737823 (+ 852us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.739309 (+ 1486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.740584 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.741349 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.741832 (+ 483us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.744646 (+ 2814us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.744649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.744651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.744676 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.746766 (+ 2090us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.747584 (+ 818us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.747590 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.747592 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.747746 (+ 154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.748063 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.748066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.748067 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.748676 (+ 609us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.749538 (+ 862us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.749687 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.750146 (+ 459us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":269,"server-negotiator.queue_time_us":321,"spinlock_wait_cycles":6144,"thread_start_us":146,"threads_started":1}
I20260807 00:04:31.751196 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.737762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:42947 (local address 127.0.0.1:46546)
0807 00:04:31.738159 (+ 397us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.738209 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.738368 (+ 159us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.738709 (+ 341us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.738714 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.739054 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.739292 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.739304 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.740329 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.740332 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.741804 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.741814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.742492 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.742496 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.742598 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.743163 (+ 565us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.743185 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.745105 (+ 1920us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.747522 (+ 2417us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.747533 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.747545 (+ 12us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.747800 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.748123 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.748126 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.748127 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.748242 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.748849 (+ 607us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.748857 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.749028 (+ 171us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.749795 (+ 767us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.750068 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":228,"mutex_wait_us":446,"spinlock_wait_cycles":14464,"thread_start_us":99,"threads_started":1}
I20260807 00:04:31.751199 20454 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.736407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42616 (local address 127.30.204.124:41987)
0807 00:04:31.736815 (+ 408us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.736822 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.736833 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:04:31.736866 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.736869 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.736909 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.737522 (+ 613us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.737669 (+ 147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.739316 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.740515 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.741353 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.741495 (+ 142us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.743872 (+ 2377us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.743880 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.743895 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.743922 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.747006 (+ 3084us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.747794 (+ 788us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.747797 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.747799 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.747881 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.748213 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.748216 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.748218 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.748673 (+ 455us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.749610 (+ 937us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.749686 (+ 76us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.750434 (+ 748us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":710,"server-negotiator.queue_time_us":348,"spinlock_wait_cycles":11392,"thread_start_us":174,"threads_started":1}
I20260807 00:04:31.751639 20318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.739715 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:42632)
0807 00:04:31.740571 (+ 856us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.740586 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.740666 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.740931 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.740934 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.741134 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.741254 (+ 120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.741262 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.742321 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.742325 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.743419 (+ 1094us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.743434 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.745739 (+ 2305us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.745744 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.745843 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.746232 (+ 389us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.746248 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.747657 (+ 1409us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.749964 (+ 2307us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.749968 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.749970 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.750230 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.750572 (+ 342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.750575 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.750577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.750669 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.751108 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.751111 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.751227 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.751315 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.751483 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":768}
I20260807 00:04:31.752758 20448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:04:31.753536 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.740238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42632 (local address 127.30.204.124:41987)
0807 00:04:31.740492 (+ 254us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.740500 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.740736 (+ 236us) server_negotiation.cc:407] Connection header received
0807 00:04:31.740768 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.740770 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.740808 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.741367 (+ 559us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.741495 (+ 128us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.742222 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.745243 (+ 3021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.745594 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.746059 (+ 465us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.747796 (+ 1737us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.747799 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.747802 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.747838 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.749864 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.750367 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.750370 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.750371 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.750456 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.750778 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.750781 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.750783 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.750993 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.753130 (+ 2137us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.753215 (+ 85us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.753342 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":196,"spinlock_wait_cycles":1792,"thread_start_us":93,"threads_started":1}
I20260807 00:04:31.756352 20210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: Bootstrap starting.
I20260807 00:04:31.757980 20314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: Bootstrap starting.
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:04:31.759268 20210 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:31.760222 20210 log.cc:824] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:31.760707 20314 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:31.761601 20314 log.cc:824] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:31.763245 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.753602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59800 (local address 127.30.204.125:42135)
0807 00:04:31.753719 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.753723 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.753922 (+ 199us) server_negotiation.cc:407] Connection header received
0807 00:04:31.754106 (+ 184us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.754110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.754160 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.754211 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.754741 (+ 530us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.755267 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.756272 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.756609 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.756717 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.759086 (+ 2369us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.759090 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.759093 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.759122 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.761218 (+ 2096us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.761729 (+ 511us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.761733 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.761735 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.761828 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.762183 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.762186 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.762188 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.762467 (+ 279us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.762547 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.762944 (+ 397us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.763085 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:04:31.763798 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.753287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:42135 (local address 127.0.0.1:59800)
0807 00:04:31.753857 (+ 570us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.753872 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.753994 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.754353 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.754356 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.754523 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.754625 (+ 102us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.754630 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.755373 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.755377 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.756133 (+ 756us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.756144 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.756748 (+ 604us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.756752 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.756851 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.757385 (+ 534us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.757399 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.758921 (+ 1522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.761342 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.761346 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.761349 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.761607 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.761955 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.761959 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.761961 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.762065 (+ 104us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.762629 (+ 564us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.762632 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.762753 (+ 121us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.763400 (+ 647us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.763595 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":262}
I20260807 00:04:31.764202 20314 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: No bootstrap required, opened a new log
I20260807 00:04:31.764204 20210 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: No bootstrap required, opened a new log
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:31.768169 20314 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.768170 20210 raft_consensus.cc:348] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.768414 20210 raft_consensus.cc:374] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:31.768415 20314 raft_consensus.cc:374] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:31.768482 20314 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9de06e6410842c8ae0595dcdb0cf649, State: Initialized, Role: FOLLOWER
I20260807 00:04:31.768482 20210 raft_consensus.cc:729] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: aae7ce3994704891897169f287d921f1, State: Initialized, Role: FOLLOWER
I20260807 00:04:31.768965 20314 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.769331 20210 consensus_queue.cc:260] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.769562 20448 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:04:31.770603 20314 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:31.770907 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.759590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46560 (local address 127.30.204.126:42947)
0807 00:04:31.759740 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.759745 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.759760 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:04:31.759806 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.759809 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.759854 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.759944 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.761096 (+ 1152us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.761776 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.762941 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.763263 (+ 322us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.763370 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.766703 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.766707 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.766711 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.766755 (+ 44us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.768430 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.769000 (+ 570us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.769004 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.769005 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.769107 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.769449 (+ 342us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.769452 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.769454 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.769734 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.769894 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.770613 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.770756 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:04:31.770890 20210 sys_catalog.cc:564] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:31.771700 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.771867 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:04:31.772408 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.772553 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [sys.catalog]: This master's current role is: FOLLOWER
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
Aug 07 00:04:31 dist-test-slave-pww0 krb5kdc[20082](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061071, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:04:31.774973 20220 master_runner.cc:186] Error getting master registration for 127.30.204.126:42947: OK, instance_id { permanent_uuid: "aae7ce3994704891897169f287d921f1" instance_seqno: 1786061071266624 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:04:31.780587 20220 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:04:31.783071 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.770098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:41987 (local address 127.0.0.1:42636)
0807 00:04:31.770836 (+ 738us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:31.770850 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:31.770939 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:31.771721 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:31.771724 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:31.772064 (+ 340us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:31.772195 (+ 131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.772203 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.773268 (+ 1065us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.773271 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.774028 (+ 757us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:31.774038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.774661 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.774664 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:31.774739 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.775432 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:31.775445 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:31.777121 (+ 1676us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:31.781405 (+ 4284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.781408 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.781411 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.781663 (+ 252us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.781931 (+ 268us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:31.781934 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:31.781936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:31.782016 (+ 80us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:31.782557 (+ 541us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:31.782559 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:31.782654 (+ 95us) client_negotiation.cc:769] Sending connection context
0807 00:04:31.782750 (+ 96us) client_negotiation.cc:240] Negotiation successful
0807 00:04:31.782907 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":685}
I20260807 00:04:31.783793 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.770380 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42636 (local address 127.30.204.124:41987)
0807 00:04:31.770763 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.770768 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.771236 (+ 468us) server_negotiation.cc:407] Connection header received
0807 00:04:31.771336 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.771339 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.771392 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.771584 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.772324 (+ 740us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.773155 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.774149 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.774453 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.774545 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.777266 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.777269 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.777272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.777297 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.781281 (+ 3984us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.781768 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.781772 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.781774 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.781851 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.782139 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.782143 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.782144 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.782432 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.782524 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.783506 (+ 982us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.783644 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246}
I20260807 00:04:31.784154 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.772310 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59802 (local address 127.30.204.125:42135)
0807 00:04:31.772424 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.772428 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.772441 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:04:31.772477 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.772480 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.772522 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.772600 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.773360 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.774086 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.775145 (+ 1059us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.775448 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.775593 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.779950 (+ 4357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.779955 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.779958 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.779993 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.781691 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.782275 (+ 584us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.782279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.782281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.782379 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.782711 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.782716 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.782717 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.782954 (+ 237us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.783165 (+ 211us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.783877 (+ 712us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.784003 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:04:31.787566 20448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764: Bootstrap starting.
W20260807 00:04:31.789393 20495 catalog_manager.cc:1558] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:04:31.789489 20495 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:04:31.790693 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:04:31.791184 20448 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:31.792290 20448 log.cc:824] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764: Log is configured to *not* fsync() on all Append() calls
W20260807 00:04:31.793082 20479 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:04:31.793170 20479 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:04:31.785429 20101 master_runner.cc:186] Error getting master registration for 127.30.204.124:41987: OK, instance_id { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" instance_seqno: 1786061071723894 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:04:31.794114 20101 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:04:31.797659 20448 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764: No bootstrap required, opened a new log
I20260807 00:04:31.801961 20448 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.802273 20448 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:31.802332 20448 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4a50fc28c13242b899b17afa805c7764, State: Initialized, Role: FOLLOWER
I20260807 00:04:31.803587 20448 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.805064 20448 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:31.814606 20216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.795018 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46572 (local address 127.30.204.126:42947)
0807 00:04:31.795143 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.795148 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.799769 (+ 4621us) server_negotiation.cc:407] Connection header received
0807 00:04:31.799893 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.799896 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.799955 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.800042 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.800993 (+ 951us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.801744 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.803080 (+ 1336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.803375 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.803451 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.810354 (+ 6903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.810358 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.810361 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.810398 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.812298 (+ 1900us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.812797 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.812801 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.812802 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.812897 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.813195 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.813198 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.813200 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.813432 (+ 232us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.813552 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.814315 (+ 763us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.814438 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
W20260807 00:04:31.817054 20534 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:04:31.817237 20534 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:04:31.817390 20323 master_runner.cc:263] No leader master found from master 4a50fc28c13242b899b17afa805c7764
I20260807 00:04:31.817469 20323 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:04:31.821395 20513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.822526 20320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.794552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59808 (local address 127.30.204.125:42135)
0807 00:04:31.794672 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.794676 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.795416 (+ 740us) server_negotiation.cc:407] Connection header received
0807 00:04:31.795700 (+ 284us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.795704 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.795760 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.795814 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.807177 (+ 11363us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.807941 (+ 764us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.809124 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.809458 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.809549 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.817744 (+ 8195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.817749 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.817751 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.817783 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.819428 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.820072 (+ 644us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.820076 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.820078 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.820174 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.820471 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.820474 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.820476 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.820730 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.822129 (+ 1399us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.822217 (+ 88us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.822356 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:04:31.824190 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:31.795581 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42644 (local address 127.30.204.124:41987)
0807 00:04:31.795709 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:31.795712 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:31.807792 (+ 12080us) server_negotiation.cc:407] Connection header received
0807 00:04:31.807984 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:31.807987 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:31.808034 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:31.808081 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:31.809058 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.809809 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.810943 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:31.811280 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:31.811348 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:31.820565 (+ 9217us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:31.820570 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:31.820573 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:31.820603 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:31.822237 (+ 1634us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.822718 (+ 481us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.822734 (+ 16us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.822736 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.822825 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:31.823128 (+ 303us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:31.823131 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:31.823133 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:31.823386 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:31.823520 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:31.823902 (+ 382us) server_negotiation.cc:299] Negotiation successful
0807 00:04:31.824031 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:04:31.824942 20513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:04:31.842689 20513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:31.842912 20513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.844071 20513 leader_election.cc:288] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers aae7ce3994704891897169f287d921f1 (127.30.204.126:42947), e9de06e6410842c8ae0595dcdb0cf649 (127.30.204.125:42135)
I20260807 00:04:31.845006 20162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a50fc28c13242b899b17afa805c7764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae7ce3994704891897169f287d921f1" is_pre_election: true
I20260807 00:04:31.845378 20162 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a50fc28c13242b899b17afa805c7764 in term 0.
I20260807 00:04:31.845854 20278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a50fc28c13242b899b17afa805c7764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9de06e6410842c8ae0595dcdb0cf649" is_pre_election: true
I20260807 00:04:31.846169 20278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4a50fc28c13242b899b17afa805c7764 in term 0.
I20260807 00:04:31.846649 20365 leader_election.cc:302] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a50fc28c13242b899b17afa805c7764, e9de06e6410842c8ae0595dcdb0cf649; no voters:
I20260807 00:04:31.847373 20513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:04:31.847483 20513 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:04:31.847533 20513 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:31.848867 20513 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.849531 20162 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a50fc28c13242b899b17afa805c7764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "aae7ce3994704891897169f287d921f1"
I20260807 00:04:31.849689 20162 raft_consensus.cc:3037] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:31.851060 20162 raft_consensus.cc:2454] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a50fc28c13242b899b17afa805c7764 in term 1.
I20260807 00:04:31.851433 20364 leader_election.cc:302] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4a50fc28c13242b899b17afa805c7764, aae7ce3994704891897169f287d921f1; no voters:
I20260807 00:04:31.852180 20278 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "4a50fc28c13242b899b17afa805c7764" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9de06e6410842c8ae0595dcdb0cf649"
I20260807 00:04:31.852317 20278 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:31.853660 20278 raft_consensus.cc:2454] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4a50fc28c13242b899b17afa805c7764 in term 1.
I20260807 00:04:31.854543 20513 leader_election.cc:288] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [CANDIDATE]: Term 1 election: Requested vote from peers aae7ce3994704891897169f287d921f1 (127.30.204.126:42947), e9de06e6410842c8ae0595dcdb0cf649 (127.30.204.125:42135)
I20260807 00:04:31.854705 20513 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:31.854800 20513 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [term 1 LEADER]: Becoming Leader. State: Replica: 4a50fc28c13242b899b17afa805c7764, State: Running, Role: LEADER
I20260807 00:04:31.855171 20513 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } }
I20260807 00:04:31.857194 20513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a50fc28c13242b899b17afa805c7764. Latest consensus state: current_term: 1 leader_uuid: "4a50fc28c13242b899b17afa805c7764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.857333 20513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:31.858035 20546 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:31.859033 20546 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:31.869900 20278 raft_consensus.cc:1261] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [term 1 FOLLOWER]: Refusing update from remote peer 4a50fc28c13242b899b17afa805c7764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:31.870759 20544 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:31.875423 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a50fc28c13242b899b17afa805c7764. Latest consensus state: current_term: 1 leader_uuid: "4a50fc28c13242b899b17afa805c7764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.877586 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:04:31.879437 20162 raft_consensus.cc:1261] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [term 1 FOLLOWER]: Refusing update from remote peer 4a50fc28c13242b899b17afa805c7764: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:31.880090 20544 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [LEADER]: Connected to new peer: Peer: permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:31.884872 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4a50fc28c13242b899b17afa805c7764. Latest consensus state: current_term: 1 leader_uuid: "4a50fc28c13242b899b17afa805c7764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.885021 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:04:31.888217 20544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a50fc28c13242b899b17afa805c7764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.888370 20544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:31.891285 20513 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "4a50fc28c13242b899b17afa805c7764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.891430 20513 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:31.894140 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a50fc28c13242b899b17afa805c7764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.894291 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:04:31.894855 20548 mvcc.cc:205] Tried to move back new op lower bound from 7315706150370095104 to 7315706150322540544. Current Snapshot: MvccSnapshot[applied={T|T < 7315706150370095104}]
I20260807 00:04:31.896353 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "4a50fc28c13242b899b17afa805c7764" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "aae7ce3994704891897169f287d921f1" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42947 } } peers { permanent_uuid: "e9de06e6410842c8ae0595dcdb0cf649" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 42135 } } peers { permanent_uuid: "4a50fc28c13242b899b17afa805c7764" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 41987 } } }
I20260807 00:04:31.896488 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:04:31.897061 20547 mvcc.cc:205] Tried to move back new op lower bound from 7315706150370095104 to 7315706150322540544. Current Snapshot: MvccSnapshot[applied={T|T < 7315706150370095104}]
I20260807 00:04:31.902897 20546 catalog_manager.cc:1347] Generated new cluster ID: 330e361956514be2b9f7e05e7c4de301
I20260807 00:04:31.902994 20546 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:04:31.937592 20546 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:04:31.939002 20546 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:04:31.966706 20546 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4a50fc28c13242b899b17afa805c7764: Generated new TSK 0
I20260807 00:04:31.967732 20546 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:04:32.791158 20495 catalog_manager.cc:1259] Loaded cluster ID: 330e361956514be2b9f7e05e7c4de301
I20260807 00:04:32.791250 20495 catalog_manager.cc:1552] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: loading cluster ID for follower catalog manager: success
I20260807 00:04:32.793157 20495 catalog_manager.cc:1574] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: acquiring CA information for follower catalog manager: success
I20260807 00:04:32.794701 20479 catalog_manager.cc:1259] Loaded cluster ID: 330e361956514be2b9f7e05e7c4de301
I20260807 00:04:32.794792 20479 catalog_manager.cc:1552] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: loading cluster ID for follower catalog manager: success
I20260807 00:04:32.795049 20495 catalog_manager.cc:1602] T 00000000000000000000000000000000 P aae7ce3994704891897169f287d921f1: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:04:32.796523 20479 catalog_manager.cc:1574] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: acquiring CA information for follower catalog manager: success
I20260807 00:04:32.797699 20479 catalog_manager.cc:1602] T 00000000000000000000000000000000 P e9de06e6410842c8ae0595dcdb0cf649: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:04:33.051674 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 20101
I20260807 00:04:33.068728 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 20220
I20260807 00:04:33.086097 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 20323
2026-08-07T00:04:33Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3325 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:04:33 dist-test-slave-pww0 krb5kdc[20570](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:04:33 dist-test-slave-pww0 krb5kdc[20570](info): set up 2 sockets
Aug 07 00:04:33 dist-test-slave-pww0 krb5kdc[20570](info): commencing operation
krb5kdc: starting...
W20260807 00:04:34.246889 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:04:34 dist-test-slave-pww0 krb5kdc[20570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:04:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:04:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:04:34.382700 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:38195
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36481
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:38195
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:34.502341 20594 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:34.502686 20594 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:34.502749 20594 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:34.506959 20594 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:34.507053 20594 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:34.507082 20594 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:34.507103 20594 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:34.507124 20594 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:34.512845 20594 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36481
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:38195
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:38195
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20594
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:34.514067 20594 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:34.515215 20594 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:34.524305 20602 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:34.524489 20594 server_base.cc:1034] running on GCE node
W20260807 00:04:34.524317 20605 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:34.524317 20600 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:34.525322 20594 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:34.526567 20594 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:34.527740 20594 hybrid_clock.cc:648] HybridClock initialized: now 1786061074527716 us; error 33 us; skew 500 ppm
Aug 07 00:04:34 dist-test-slave-pww0 krb5kdc[20570](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061074, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:34.530714 20594 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:34.531822 20594 webserver.cc:460] Webserver started at http://127.30.204.126:44769/ using document root <none> and password file <none>
I20260807 00:04:34.532426 20594 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:34.532503 20594 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:34.532727 20594 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:34.534955 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "e46804a0d3c048a984db056fed509143"
format_stamp: "Formatted at 2026-08-07 00:04:34 on dist-test-slave-pww0"
server_key: "6cdd61472e5995cafd6696b9f3a336a5"
server_key_iv: "d0b174ea90966a00b90af29fbb08a7d8"
server_key_version: "encryptionkey@0"
I20260807 00:04:34.535609 20594 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "e46804a0d3c048a984db056fed509143"
format_stamp: "Formatted at 2026-08-07 00:04:34 on dist-test-slave-pww0"
server_key: "6cdd61472e5995cafd6696b9f3a336a5"
server_key_iv: "d0b174ea90966a00b90af29fbb08a7d8"
server_key_version: "encryptionkey@0"
I20260807 00:04:34.539624 20594 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:04:34.542352 20617 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:34.543735 20594 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:04:34.543870 20594 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "e46804a0d3c048a984db056fed509143"
format_stamp: "Formatted at 2026-08-07 00:04:34 on dist-test-slave-pww0"
server_key: "6cdd61472e5995cafd6696b9f3a336a5"
server_key_iv: "d0b174ea90966a00b90af29fbb08a7d8"
server_key_version: "encryptionkey@0"
I20260807 00:04:34.543985 20594 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:34.570394 20594 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:34.574039 20594 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:34.574257 20594 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:34.581872 20594 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:38195
I20260807 00:04:34.581913 20698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:38195 every 8 connection(s)
I20260807 00:04:34.582890 20594 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:34.585917 20699 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:34.590939 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 20594
I20260807 00:04:34.591116 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:04:34.591440 31537 external_mini_cluster.cc:1428] Setting key 46f74b6d0473bfe0d74cbc93d9891c8f
I20260807 00:04:34.592509 20699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143: Bootstrap starting.
I20260807 00:04:34.595052 20699 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:34.596016 20699 log.cc:824] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:34.598487 20699 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143: No bootstrap required, opened a new log
Aug 07 00:04:34 dist-test-slave-pww0 krb5kdc[20570](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061074, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:34.602965 20699 raft_consensus.cc:348] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46804a0d3c048a984db056fed509143" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38195 } }
I20260807 00:04:34.603314 20699 raft_consensus.cc:374] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:34.603381 20699 raft_consensus.cc:729] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e46804a0d3c048a984db056fed509143, State: Initialized, Role: FOLLOWER
I20260807 00:04:34.603941 20699 consensus_queue.cc:260] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46804a0d3c048a984db056fed509143" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38195 } }
I20260807 00:04:34.604087 20699 raft_consensus.cc:388] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:04:34.604179 20699 raft_consensus.cc:482] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:04:34.604293 20699 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:34.605402 20699 raft_consensus.cc:504] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46804a0d3c048a984db056fed509143" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38195 } }
I20260807 00:04:34.605808 20699 leader_election.cc:302] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e46804a0d3c048a984db056fed509143; no voters:
I20260807 00:04:34.606209 20699 leader_election.cc:288] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:04:34.606326 20709 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:34.606534 20709 raft_consensus.cc:686] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [term 1 LEADER]: Becoming Leader. State: Replica: e46804a0d3c048a984db056fed509143, State: Running, Role: LEADER
I20260807 00:04:34.606951 20709 consensus_queue.cc:237] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46804a0d3c048a984db056fed509143" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38195 } }
I20260807 00:04:34.607501 20706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:34.592975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59158 (local address 127.30.204.126:38195)
0807 00:04:34.593425 (+ 450us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:34.593441 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:34.593478 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:04:34.594154 (+ 676us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:34.594171 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:34.594516 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:34.594816 (+ 300us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:34.595814 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:34.597083 (+ 1269us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:34.598274 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:34.598852 (+ 578us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:34.598980 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:34.602060 (+ 3080us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:34.602066 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:34.602082 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:34.602116 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:34.604061 (+ 1945us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:34.604559 (+ 498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:34.604566 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:34.604573 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:34.604700 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:34.604978 (+ 278us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:34.604980 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:34.604981 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:34.605409 (+ 428us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:34.605583 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:34.605935 (+ 352us) server_negotiation.cc:299] Negotiation successful
0807 00:04:34.606297 (+ 362us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":300,"thread_start_us":132,"threads_started":1}
I20260807 00:04:34.607503 20699 sys_catalog.cc:564] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:34.608469 20715 sys_catalog.cc:455] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e46804a0d3c048a984db056fed509143. Latest consensus state: current_term: 1 leader_uuid: "e46804a0d3c048a984db056fed509143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46804a0d3c048a984db056fed509143" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38195 } } }
I20260807 00:04:34.608592 20715 sys_catalog.cc:458] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:34.609339 20712 sys_catalog.cc:455] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e46804a0d3c048a984db056fed509143" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e46804a0d3c048a984db056fed509143" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38195 } } }
I20260807 00:04:34.609480 20712 sys_catalog.cc:458] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:34.610051 20726 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:34.611188 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:04:34.613756 20726 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:34.619606 20726 catalog_manager.cc:1347] Generated new cluster ID: 47b970e2f2d64001a14b71b3b6d0e2f7
I20260807 00:04:34.619684 20726 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:04:34.628072 20726 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:04:34.629148 20726 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:04:34.635480 20726 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e46804a0d3c048a984db056fed509143: Generated new TSK 0
I20260807 00:04:34.636291 20726 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:04:34.667703 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 20594
2026-08-07T00:04:34Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1590 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:04:34 dist-test-slave-pww0 krb5kdc[20752](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:04:34 dist-test-slave-pww0 krb5kdc[20752](info): set up 2 sockets
Aug 07 00:04:34 dist-test-slave-pww0 krb5kdc[20752](info): commencing operation
krb5kdc: starting...
W20260807 00:04:35.844164 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:04:35 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:04:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:04:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:04:35.980012 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41003
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:41003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:36.098186 20771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:36.098479 20771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:36.098523 20771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:36.102800 20771 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:36.102890 20771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:36.102908 20771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:36.102926 20771 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:36.102939 20771 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:36.108703 20771 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:41003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:36.109902 20771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:36.111048 20771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:04:36.120432 20771 server_base.cc:1034] running on GCE node
W20260807 00:04:36.120375 20779 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:36.120375 20782 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:36.120379 20780 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:36.121338 20771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:36.122581 20771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:36.123760 20771 hybrid_clock.cc:648] HybridClock initialized: now 1786061076123738 us; error 32 us; skew 500 ppm
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:36.130149 20771 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:36.131253 20771 webserver.cc:460] Webserver started at http://127.30.204.126:43971/ using document root <none> and password file <none>
I20260807 00:04:36.131891 20771 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:36.131966 20771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:36.132190 20771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:36.134413 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "086ddf14dfc541cb8e374c89ad462290"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "e25253e62c3137b7864e39c1addde0c6"
server_key_iv: "b6663a42f127c9c6038b83fcd4b7c2f9"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.135001 20771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "086ddf14dfc541cb8e374c89ad462290"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "e25253e62c3137b7864e39c1addde0c6"
server_key_iv: "b6663a42f127c9c6038b83fcd4b7c2f9"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.139127 20771 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 00:04:36.141960 20791 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.143405 20771 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:04:36.143582 20771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "086ddf14dfc541cb8e374c89ad462290"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "e25253e62c3137b7864e39c1addde0c6"
server_key_iv: "b6663a42f127c9c6038b83fcd4b7c2f9"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.143705 20771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:36.167379 20771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:36.171271 20771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:36.171527 20771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:36.179636 20771 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:41003
I20260807 00:04:36.179921 20873 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:41003 every 8 connection(s)
I20260807 00:04:36.180930 20771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:36.184118 20874 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:36.188702 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 20771
I20260807 00:04:36.188870 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:04:36.189186 31537 external_mini_cluster.cc:1428] Setting key c87879cc061b1d9dac6413eb87f7caec
I20260807 00:04:36.191027 20874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Bootstrap starting.
I20260807 00:04:36.193753 20874 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:36.194535 20874 log.cc:824] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:36.196898 20874 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: No bootstrap required, opened a new log
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061075, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:36.200845 20874 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:36.201181 20874 raft_consensus.cc:374] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:36.201249 20874 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ddf14dfc541cb8e374c89ad462290, State: Initialized, Role: FOLLOWER
I20260807 00:04:36.201741 20874 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:36.201886 20874 raft_consensus.cc:388] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:04:36.202009 20874 raft_consensus.cc:482] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:04:36.202140 20874 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:36.203254 20874 raft_consensus.cc:504] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:36.203722 20874 leader_election.cc:302] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 086ddf14dfc541cb8e374c89ad462290; no voters:
I20260807 00:04:36.204171 20874 leader_election.cc:288] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:04:36.204289 20888 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:36.204523 20888 raft_consensus.cc:686] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 LEADER]: Becoming Leader. State: Replica: 086ddf14dfc541cb8e374c89ad462290, State: Running, Role: LEADER
I20260807 00:04:36.204922 20888 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:36.205101 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.190813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40538 (local address 127.30.204.126:41003)
0807 00:04:36.191281 (+ 468us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:36.191299 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:36.191339 (+ 40us) server_negotiation.cc:407] Connection header received
0807 00:04:36.192069 (+ 730us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:36.192088 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:36.192457 (+ 369us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:36.192767 (+ 310us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:36.193693 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.194838 (+ 1145us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.195953 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.196464 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.196620 (+ 156us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:36.199560 (+ 2940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:36.199566 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:36.199583 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:36.199617 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:36.201849 (+ 2232us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.202369 (+ 520us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.202375 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.202380 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.202502 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.202786 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.202788 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.202789 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.203197 (+ 408us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:36.203346 (+ 149us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:36.203694 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 00:04:36.204001 (+ 307us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":311,"thread_start_us":127,"threads_started":1}
I20260807 00:04:36.205509 20874 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:36.206295 20888 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086ddf14dfc541cb8e374c89ad462290. Latest consensus state: current_term: 1 leader_uuid: "086ddf14dfc541cb8e374c89ad462290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } } }
I20260807 00:04:36.206481 20888 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:36.207186 20890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "086ddf14dfc541cb8e374c89ad462290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } } }
I20260807 00:04:36.207311 20890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:36.207486 20909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:36.210552 20909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:36.216591 20909 catalog_manager.cc:1347] Generated new cluster ID: 74f02c21bac1462c80b5caf644a1b80c
I20260807 00:04:36.216681 20909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:04:36.242611 20909 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:04:36.243947 20909 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:04:36.248811 20909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Generated new TSK 0
I20260807 00:04:36.249677 20909 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:04:36.280910 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41003
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:36.405344 20927 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:36.405694 20927 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:36.405758 20927 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:36.410177 20927 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:36.410284 20927 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:36.410375 20927 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:04:36.416095 20927 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20927
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:36.417367 20927 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:36.418582 20927 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:36.429201 20942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:36.429260 20936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:36.429469 20927 server_base.cc:1034] running on GCE node
W20260807 00:04:36.429718 20934 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:36.430188 20927 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:36.430965 20927 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:36.432149 20927 hybrid_clock.cc:648] HybridClock initialized: now 1786061076432134 us; error 30 us; skew 500 ppm
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:36.435196 20927 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:04:36.436259 20927 webserver.cc:460] Webserver started at http://127.30.204.65:35967/ using document root <none> and password file <none>
I20260807 00:04:36.436872 20927 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:36.436949 20927 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:36.437168 20927 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:36.439395 20927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "5391577b946d404db61ea40c0b93367e"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "6568aa1dc3fe4e072e86c05bfbc1470e"
server_key_iv: "4e48bca151840fed644bb5f35d2fe54a"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.440045 20927 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "5391577b946d404db61ea40c0b93367e"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "6568aa1dc3fe4e072e86c05bfbc1470e"
server_key_iv: "4e48bca151840fed644bb5f35d2fe54a"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.444151 20927 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:04:36.446905 20955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.448405 20927 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.003s
I20260807 00:04:36.448531 20927 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "5391577b946d404db61ea40c0b93367e"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "6568aa1dc3fe4e072e86c05bfbc1470e"
server_key_iv: "4e48bca151840fed644bb5f35d2fe54a"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.448668 20927 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:36.467139 20927 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:36.470623 20927 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:36.470840 20927 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:36.471558 20927 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:36.472478 20927 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:36.472529 20927 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.472586 20927 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:36.472613 20927 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.482501 20927 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:41535
I20260807 00:04:36.482542 21146 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:41535 every 8 connection(s)
I20260807 00:04:36.483539 20927 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:04:36.489573 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 20927
I20260807 00:04:36.489727 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:04:36.490010 31537 external_mini_cluster.cc:1428] Setting key 4f428037e9d4642d04acea71d1eb6d24
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:36.498960 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.485598 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:59263 (local address 127.30.204.126:41003)
0807 00:04:36.485777 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:36.485781 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:36.486752 (+ 971us) server_negotiation.cc:407] Connection header received
0807 00:04:36.488218 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:36.488221 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:36.488278 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:36.488361 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:36.489674 (+ 1313us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.490345 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.491816 (+ 1471us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.492043 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.492143 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:36.495026 (+ 2883us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:36.495029 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:36.495031 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:36.495074 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:36.496508 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.497105 (+ 597us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.497109 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.497111 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.497220 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.497637 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.497640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.497642 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.497901 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:36.498014 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:36.498620 (+ 606us) server_negotiation.cc:299] Negotiation successful
0807 00:04:36.498773 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:04:36.499862 21155 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.486012 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:59263)
0807 00:04:36.486577 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:36.486627 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:36.487968 (+ 1341us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:36.488534 (+ 566us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:36.488541 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:36.488972 (+ 431us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:36.489493 (+ 521us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:36.489508 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.490492 (+ 984us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.490495 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:36.491698 (+ 1203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:36.491710 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.492206 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.492210 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:36.492393 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:36.492986 (+ 593us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:36.493005 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:36.494789 (+ 1784us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:36.496652 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:36.496660 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:36.496681 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:36.496979 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:36.497351 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:36.497355 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:36.497357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:36.497533 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:36.498038 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:36.498044 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:36.498348 (+ 304us) client_negotiation.cc:769] Sending connection context
0807 00:04:36.498533 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 00:04:36.498845 (+ 312us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":133,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:04:36.501361 21147 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41003
I20260807 00:04:36.501626 21147 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:36.502277 21147 heartbeater.cc:511] Master 127.30.204.126:41003 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:04:36.504417 20815 ts_manager.cc:194] Registered new tserver with Master: 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535)
I20260807 00:04:36.506021 20815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:59263
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:04:36.523093 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41003
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:36.641714 21159 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:36.642004 21159 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:36.642048 21159 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:36.646306 21159 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:36.646384 21159 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:36.646448 21159 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:04:36.652163 21159 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21159
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:36.653394 21159 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:36.654531 21159 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:36.665512 21166 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:36.665575 21167 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:36.665544 21171 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:36.665927 21159 server_base.cc:1034] running on GCE node
I20260807 00:04:36.666409 21159 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:36.667128 21159 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:36.668329 21159 hybrid_clock.cc:648] HybridClock initialized: now 1786061076668280 us; error 60 us; skew 500 ppm
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:36.671411 21159 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:04:36.672502 21159 webserver.cc:460] Webserver started at http://127.30.204.66:46701/ using document root <none> and password file <none>
I20260807 00:04:36.673097 21159 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:36.673164 21159 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:36.673367 21159 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:36.675707 21159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "c1e17142eb5f4939a2ea75a920fbbe6a"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "ebf4c6cf7a966b6d1aec0e62f9facd61"
server_key_iv: "7799eebccddfc6d2bd91a5cf52bbec58"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.676295 21159 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "c1e17142eb5f4939a2ea75a920fbbe6a"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "ebf4c6cf7a966b6d1aec0e62f9facd61"
server_key_iv: "7799eebccddfc6d2bd91a5cf52bbec58"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.680351 21159 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:04:36.683145 21181 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.684546 21159 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:04:36.684679 21159 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "c1e17142eb5f4939a2ea75a920fbbe6a"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "ebf4c6cf7a966b6d1aec0e62f9facd61"
server_key_iv: "7799eebccddfc6d2bd91a5cf52bbec58"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.684808 21159 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:36.700299 21159 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:36.704010 21159 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:36.704252 21159 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:36.704907 21159 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:36.705866 21159 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:36.705919 21159 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.705974 21159 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:36.706000 21159 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.715348 21159 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:35349
I20260807 00:04:36.715387 21357 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:35349 every 8 connection(s)
I20260807 00:04:36.716370 21159 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:04:36.721959 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 21159
I20260807 00:04:36.722091 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:04:36.722383 31537 external_mini_cluster.cc:1428] Setting key c1deece550bc414730c62448d3d0e74b
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:36.731983 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.718469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:45699 (local address 127.30.204.126:41003)
0807 00:04:36.718599 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:36.718602 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:36.719580 (+ 978us) server_negotiation.cc:407] Connection header received
0807 00:04:36.721000 (+ 1420us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:36.721004 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:36.721066 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:36.721156 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:36.722516 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.723215 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.724658 (+ 1443us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.724946 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.725055 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:36.728023 (+ 2968us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:36.728027 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:36.728030 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:36.728061 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:36.729649 (+ 1588us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.730195 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.730200 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.730202 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.730316 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.730667 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.730670 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.730672 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.730931 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:36.731038 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:36.731620 (+ 582us) server_negotiation.cc:299] Negotiation successful
0807 00:04:36.731771 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:04:36.732856 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.718879 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:45699)
0807 00:04:36.719396 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:36.719444 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:36.720773 (+ 1329us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:36.721368 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:36.721376 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:36.721759 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:36.722337 (+ 578us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:36.722353 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.723361 (+ 1008us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.723365 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:36.724520 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:36.724531 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.725148 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.725152 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:36.725335 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:36.726004 (+ 669us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:36.726032 (+ 28us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:36.727826 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:36.729773 (+ 1947us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:36.729779 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:36.729794 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:36.730075 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:36.730416 (+ 341us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:36.730418 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:36.730420 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:36.730568 (+ 148us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:36.731024 (+ 456us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:36.731029 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:36.731318 (+ 289us) client_negotiation.cc:769] Sending connection context
0807 00:04:36.731554 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 00:04:36.731869 (+ 315us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":136,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:04:36.734129 21362 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41003
I20260807 00:04:36.734422 21362 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:36.734977 21362 heartbeater.cc:511] Master 127.30.204.126:41003 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:04:36.736236 20815 ts_manager.cc:194] Registered new tserver with Master: c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349)
I20260807 00:04:36.736832 20815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:45699
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:04:36.754488 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41003
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:36.878186 21372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:36.878484 21372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:36.878530 21372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:36.882710 21372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:36.882783 21372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:36.882849 21372 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:04:36.888542 21372 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:36.889796 21372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:36.890906 21372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:36.901006 21382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:36.901012 21385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:36.901007 21383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:36.901332 21372 server_base.cc:1034] running on GCE node
I20260807 00:04:36.901782 21372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:36.902431 21372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:36.903592 21372 hybrid_clock.cc:648] HybridClock initialized: now 1786061076903564 us; error 41 us; skew 500 ppm
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:36.906662 21372 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:04:36.907688 21372 webserver.cc:460] Webserver started at http://127.30.204.67:43323/ using document root <none> and password file <none>
I20260807 00:04:36.908285 21372 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:36.908351 21372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:36.908591 21372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:36.910758 21372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "b821bd8f67d143b480be20f116491df0"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "f454ad88d2da9d5bfd2e20a34787894c"
server_key_iv: "dbe198cce6c5d3485b8a41c89b638040"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.911351 21372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "b821bd8f67d143b480be20f116491df0"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "f454ad88d2da9d5bfd2e20a34787894c"
server_key_iv: "dbe198cce6c5d3485b8a41c89b638040"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.915450 21372 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.003s
I20260807 00:04:36.918319 21395 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.919755 21372 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:04:36.919873 21372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "b821bd8f67d143b480be20f116491df0"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "f454ad88d2da9d5bfd2e20a34787894c"
server_key_iv: "dbe198cce6c5d3485b8a41c89b638040"
server_key_version: "encryptionkey@0"
I20260807 00:04:36.919991 21372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:36.941273 21372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:36.944981 21372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:36.945237 21372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:36.946103 21372 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:36.947324 21372 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:36.947402 21372 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.947503 21372 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:36.947536 21372 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:36.961405 21372 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:43351
I20260807 00:04:36.961431 21580 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:43351 every 8 connection(s)
I20260807 00:04:36.962811 21372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:04:36.964437 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 21372
I20260807 00:04:36.964581 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:04:36.964912 31537 external_mini_cluster.cc:1428] Setting key de7e87a2f8f0b771d7040a896dada366
Aug 07 00:04:36 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061076, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:36.985559 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.967674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:60133 (local address 127.30.204.126:41003)
0807 00:04:36.967808 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:36.967813 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:36.967983 (+ 170us) server_negotiation.cc:407] Connection header received
0807 00:04:36.969754 (+ 1771us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:36.969758 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:36.969823 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:36.969900 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:36.971965 (+ 2065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.972828 (+ 863us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.974914 (+ 2086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.975286 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.975378 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:36.980821 (+ 5443us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:36.980825 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:36.980827 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:36.980865 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:36.982611 (+ 1746us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.983424 (+ 813us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.983428 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.983430 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.983574 (+ 144us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:36.984036 (+ 462us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:36.984040 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:36.984042 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:36.984324 (+ 282us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:36.984436 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:36.985237 (+ 801us) server_negotiation.cc:299] Negotiation successful
0807 00:04:36.985389 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:04:36.986737 21584 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.967007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:60133)
0807 00:04:36.967719 (+ 712us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:36.967786 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:36.969513 (+ 1727us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:36.970280 (+ 767us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:36.970292 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:36.970875 (+ 583us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:36.971735 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:36.971764 (+ 29us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.973043 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.973047 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:36.974709 (+ 1662us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:36.974740 (+ 31us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.975695 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.975700 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:36.975975 (+ 275us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:36.976712 (+ 737us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:36.976749 (+ 37us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:36.980590 (+ 3841us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:36.982816 (+ 2226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:36.982826 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:36.982849 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:36.983286 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:36.983696 (+ 410us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:36.983699 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:36.983701 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:36.983918 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:36.984437 (+ 519us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:36.984446 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:36.984954 (+ 508us) client_negotiation.cc:769] Sending connection context
0807 00:04:36.985165 (+ 211us) client_negotiation.cc:240] Negotiation successful
0807 00:04:36.985555 (+ 390us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":438,"thread_start_us":181,"threads_started":1}
I20260807 00:04:36.988298 21581 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41003
I20260807 00:04:36.988580 21581 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:36.989477 21581 heartbeater.cc:511] Master 127.30.204.126:41003 requested a full tablet report, sending...
I20260807 00:04:36.991640 20815 ts_manager.cc:194] Registered new tserver with Master: b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351)
I20260807 00:04:36.992472 20815 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:60133
I20260807 00:04:36.993932 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:04:37.004753 20882 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:36.996465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40542 (local address 127.30.204.126:41003)
0807 00:04:36.996598 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:36.996603 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:36.996853 (+ 250us) server_negotiation.cc:407] Connection header received
0807 00:04:36.996975 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:36.996978 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:36.997026 (+ 48us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:36.997091 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:36.997817 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.998339 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.999165 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:36.999394 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:36.999511 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.000835 (+ 1324us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:37.000838 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:37.000840 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:37.000891 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:37.002519 (+ 1628us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.003002 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.003006 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.003009 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.003154 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.003504 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.003506 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.003507 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.003881 (+ 374us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:37.004002 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.004380 (+ 378us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.004592 (+ 212us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260807 00:04:37.008314 20815 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40542:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:04:37.012048 20815 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:04:37.029016 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.022891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52074 (local address 127.30.204.67:43351)
0807 00:04:37.023263 (+ 372us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.023275 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.023300 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:04:37.023384 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.023390 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.023658 (+ 268us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.024324 (+ 666us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:37.024484 (+ 160us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.025598 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.027315 (+ 1717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.028221 (+ 906us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.028338 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.028502 (+ 164us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.028737 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.028852 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":267,"thread_start_us":154,"threads_started":1}
I20260807 00:04:37.029008 21604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.022113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:43351 (local address 127.0.0.1:52074)
0807 00:04:37.022831 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.022884 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.023022 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.023838 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.023844 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.023866 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:37.024145 (+ 279us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.024157 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.025765 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.025769 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.027160 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.027170 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.028350 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.028354 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.028488 (+ 134us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.028536 (+ 48us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.028671 (+ 135us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.028829 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":538,"mutex_wait_us":26,"thread_start_us":96,"threads_started":1}
I20260807 00:04:37.029462 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.022364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54816 (local address 127.30.204.65:41535)
0807 00:04:37.022662 (+ 298us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.022670 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.023621 (+ 951us) server_negotiation.cc:407] Connection header received
0807 00:04:37.024380 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.024387 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.024608 (+ 221us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.024765 (+ 157us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:37.025159 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.026005 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.027808 (+ 1803us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.028609 (+ 801us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.028713 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.028855 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.029205 (+ 350us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.029318 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":180,"thread_start_us":80,"threads_started":1}
I20260807 00:04:37.029901 21605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.022185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41535 (local address 127.0.0.1:54816)
0807 00:04:37.023499 (+ 1314us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.023513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.023707 (+ 194us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.024859 (+ 1152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.024862 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.024881 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:37.025010 (+ 129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.025016 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.026141 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.026144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.027658 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.027670 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.028922 (+ 1252us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.028926 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.029021 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.029046 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.029658 (+ 612us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.029749 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1211,"mutex_wait_us":31,"thread_start_us":88,"threads_started":1}
I20260807 00:04:37.031312 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.023298 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35349 (local address 127.0.0.1:48294)
0807 00:04:37.023657 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.023671 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.023760 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.024767 (+ 1007us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.024770 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.024791 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:37.024917 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.024923 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.026547 (+ 1624us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.026551 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.029754 (+ 3203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.029766 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.030914 (+ 1148us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.030918 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.031032 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.031058 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.031106 (+ 48us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.031182 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":245,"mutex_wait_us":24,"thread_start_us":117,"threads_started":1}
I20260807 00:04:37.032164 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.023905 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48294 (local address 127.30.204.66:35349)
0807 00:04:37.024281 (+ 376us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.024293 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.024315 (+ 22us) server_negotiation.cc:407] Connection header received
0807 00:04:37.024390 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.024395 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.024589 (+ 194us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.025086 (+ 497us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:37.025265 (+ 179us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.026364 (+ 1099us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.029890 (+ 3526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.030771 (+ 881us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.031659 (+ 888us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.031828 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.031885 (+ 57us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.031990 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":264,"thread_start_us":136,"threads_started":1}
I20260807 00:04:37.033748 21461 tablet_service.cc:1467] Processing CreateTablet for tablet 5a31d87ef9724fc9b77ed174b684ee9b (DEFAULT_TABLE table=test-table [id=51256bffdd3e47068ee490b2ae30255f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:37.033524 21032 tablet_service.cc:1467] Processing CreateTablet for tablet 5a31d87ef9724fc9b77ed174b684ee9b (DEFAULT_TABLE table=test-table [id=51256bffdd3e47068ee490b2ae30255f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:37.034947 21032 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a31d87ef9724fc9b77ed174b684ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:37.034963 21461 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a31d87ef9724fc9b77ed174b684ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:37.038352 21254 tablet_service.cc:1467] Processing CreateTablet for tablet 5a31d87ef9724fc9b77ed174b684ee9b (DEFAULT_TABLE table=test-table [id=51256bffdd3e47068ee490b2ae30255f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:37.039758 21254 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 5a31d87ef9724fc9b77ed174b684ee9b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:37.042829 21610 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Bootstrap starting.
I20260807 00:04:37.043923 21611 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Bootstrap starting.
I20260807 00:04:37.046370 21611 tablet_bootstrap.cc:654] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:37.047362 21610 tablet_bootstrap.cc:654] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:37.047547 21611 log.cc:824] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:37.049397 21610 log.cc:824] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:37.050179 21611 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: No bootstrap required, opened a new log
I20260807 00:04:37.050618 21611 ts_tablet_manager.cc:1397] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:04:37.055215 21610 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: No bootstrap required, opened a new log
I20260807 00:04:37.055521 21610 ts_tablet_manager.cc:1397] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Time spent bootstrapping tablet: real 0.013s user 0.007s sys 0.000s
I20260807 00:04:37.056437 21611 raft_consensus.cc:348] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.056772 21611 raft_consensus.cc:374] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:37.056836 21611 raft_consensus.cc:729] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5391577b946d404db61ea40c0b93367e, State: Initialized, Role: FOLLOWER
I20260807 00:04:37.057510 21611 consensus_queue.cc:260] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.058357 21147 heartbeater.cc:503] Master 127.30.204.126:41003 was elected leader, sending a full tablet report...
I20260807 00:04:37.058614 21614 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Bootstrap starting.
I20260807 00:04:37.059675 21610 raft_consensus.cc:348] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.060014 21610 raft_consensus.cc:374] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:37.060120 21610 raft_consensus.cc:729] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b821bd8f67d143b480be20f116491df0, State: Initialized, Role: FOLLOWER
I20260807 00:04:37.060672 21610 consensus_queue.cc:260] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.061983 21610 ts_tablet_manager.cc:1428] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:04:37.061997 21614 tablet_bootstrap.cc:654] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:37.062398 21581 heartbeater.cc:503] Master 127.30.204.126:41003 was elected leader, sending a full tablet report...
I20260807 00:04:37.063017 21614 log.cc:824] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:37.065575 21614 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: No bootstrap required, opened a new log
I20260807 00:04:37.065893 21614 ts_tablet_manager.cc:1397] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Time spent bootstrapping tablet: real 0.008s user 0.007s sys 0.000s
I20260807 00:04:37.066696 21611 ts_tablet_manager.cc:1428] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Time spent starting tablet: real 0.016s user 0.009s sys 0.000s
I20260807 00:04:37.070312 21614 raft_consensus.cc:348] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.070719 21614 raft_consensus.cc:374] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:37.070801 21614 raft_consensus.cc:729] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e17142eb5f4939a2ea75a920fbbe6a, State: Initialized, Role: FOLLOWER
I20260807 00:04:37.071334 21614 consensus_queue.cc:260] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.072137 21362 heartbeater.cc:503] Master 127.30.204.126:41003 was elected leader, sending a full tablet report...
I20260807 00:04:37.072551 21614 ts_tablet_manager.cc:1428] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Time spent starting tablet: real 0.007s user 0.002s sys 0.003s
I20260807 00:04:37.150831 21622 raft_consensus.cc:482] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:37.151062 21622 raft_consensus.cc:504] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.152437 21622 leader_election.cc:288] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351), 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535)
I20260807 00:04:37.156158 21367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.152695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:43351 (local address 127.30.204.66:35495)
0807 00:04:37.152822 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.152837 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.152937 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.153169 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.153172 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.153200 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:37.153309 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.153314 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.154238 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.154241 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.155028 (+ 787us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.155038 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.155726 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.155730 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.155840 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.155858 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.155925 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.156009 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39}
I20260807 00:04:37.156158 21607 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.152828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:35495 (local address 127.30.204.67:43351)
0807 00:04:37.152963 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.152968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.152984 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:04:37.153039 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.153042 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.153080 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.153141 (+ 61us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:37.153436 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.154136 (+ 700us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.155153 (+ 1017us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.155602 (+ 449us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.155719 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.155779 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.155952 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.156028 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:04:37.156636 21606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.152917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:45001 (local address 127.30.204.65:41535)
0807 00:04:37.153055 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.153060 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.153517 (+ 457us) server_negotiation.cc:407] Connection header received
0807 00:04:37.153577 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.153579 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.153623 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.153713 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:37.154001 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.154508 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.155606 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.156047 (+ 441us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.156160 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.156215 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.156423 (+ 208us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.156512 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 00:04:37.156996 21624 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.152707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41535 (local address 127.30.204.66:45001)
0807 00:04:37.153357 (+ 650us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.153371 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.153483 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.153734 (+ 251us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.153736 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.153755 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:37.153868 (+ 113us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.153874 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.154636 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.154639 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.155446 (+ 807us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.155491 (+ 45us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.156160 (+ 669us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.156164 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.156272 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.156296 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.156797 (+ 501us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.156884 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":572,"mutex_wait_us":265,"thread_start_us":129,"threads_started":1}
I20260807 00:04:37.156931 21487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31d87ef9724fc9b77ed174b684ee9b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b821bd8f67d143b480be20f116491df0" is_pre_election: true
I20260807 00:04:37.157312 21487 raft_consensus.cc:2454] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 0.
I20260807 00:04:37.157634 21067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31d87ef9724fc9b77ed174b684ee9b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5391577b946d404db61ea40c0b93367e" is_pre_election: true
I20260807 00:04:37.157804 21186 leader_election.cc:302] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a; no voters:
I20260807 00:04:37.157951 21067 raft_consensus.cc:2454] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 0.
I20260807 00:04:37.158113 21622 raft_consensus.cc:2781] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:04:37.158233 21622 raft_consensus.cc:482] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:04:37.158293 21622 raft_consensus.cc:3037] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:37.159430 21622 raft_consensus.cc:504] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.159854 21622 leader_election.cc:288] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 election: Requested vote from peers b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351), 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535)
I20260807 00:04:37.160324 21487 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31d87ef9724fc9b77ed174b684ee9b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b821bd8f67d143b480be20f116491df0"
I20260807 00:04:37.160410 21067 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "5a31d87ef9724fc9b77ed174b684ee9b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5391577b946d404db61ea40c0b93367e"
I20260807 00:04:37.160480 21487 raft_consensus.cc:3037] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:37.160532 21067 raft_consensus.cc:3037] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:37.161681 21067 raft_consensus.cc:2454] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 1.
I20260807 00:04:37.161702 21487 raft_consensus.cc:2454] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 1.
I20260807 00:04:37.162031 21186 leader_election.cc:302] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5391577b946d404db61ea40c0b93367e, b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a; no voters:
I20260807 00:04:37.162227 21622 raft_consensus.cc:2781] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:37.162487 21622 raft_consensus.cc:686] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 LEADER]: Becoming Leader. State: Replica: c1e17142eb5f4939a2ea75a920fbbe6a, State: Running, Role: LEADER
I20260807 00:04:37.162875 21622 consensus_queue.cc:237] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.166549 20814 catalog_manager.cc:5495] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66). New cstate: current_term: 1 leader_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } health_report { overall_health: HEALTHY } } }
I20260807 00:04:37.189204 21609 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.184694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48310 (local address 127.30.204.66:35349)
0807 00:04:37.184838 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.184843 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.184884 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:04:37.184963 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.184966 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.185042 (+ 76us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.185147 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:04:37.185384 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.186164 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.187238 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.187696 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.187809 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.188142 (+ 333us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:04:37.188695 (+ 553us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:04:37.188811 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.188985 (+ 174us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.189052 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:04:37.195828 21067 raft_consensus.cc:1261] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Refusing update from remote peer c1e17142eb5f4939a2ea75a920fbbe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:37.195828 21487 raft_consensus.cc:1261] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Refusing update from remote peer c1e17142eb5f4939a2ea75a920fbbe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:37.196628 21625 consensus_queue.cc:1035] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:37.196833 21622 consensus_queue.cc:1035] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:37.214717 20815 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:37.214901 20815 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:40542:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:37.217669 20815 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Sending DeleteTablet for 3 replicas of tablet 5a31d87ef9724fc9b77ed174b684ee9b
I20260807 00:04:37.218617 21461 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a31d87ef9724fc9b77ed174b684ee9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:52074
I20260807 00:04:37.218829 21254 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a31d87ef9724fc9b77ed174b684ee9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:48294
I20260807 00:04:37.219281 21647 tablet_replica.cc:330] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: stopping tablet replica
I20260807 00:04:37.219347 21032 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a31d87ef9724fc9b77ed174b684ee9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:54816
I20260807 00:04:37.219435 21649 tablet_replica.cc:330] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: stopping tablet replica
I20260807 00:04:37.219489 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 20927
I20260807 00:04:37.230844 21649 raft_consensus.cc:2229] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:04:37.231519 21649 raft_consensus.cc:2258] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:37.234858 21647 raft_consensus.cc:2229] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:04:37.235406 21647 raft_consensus.cc:2258] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:37.235839 21649 ts_tablet_manager.cc:1905] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:04:37.238721 21649 ts_tablet_manager.cc:1918] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:04:37.238828 21649 log.cc:1196] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/5a31d87ef9724fc9b77ed174b684ee9b
I20260807 00:04:37.239149 21649 ts_tablet_manager.cc:1939] T 5a31d87ef9724fc9b77ed174b684ee9b P c1e17142eb5f4939a2ea75a920fbbe6a: Deleting consensus metadata
I20260807 00:04:37.240119 21647 ts_tablet_manager.cc:1905] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:04:37.240609 20794 catalog_manager.cc:4841] TS c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349): tablet 5a31d87ef9724fc9b77ed174b684ee9b (table test-table [id=51256bffdd3e47068ee490b2ae30255f]) successfully deleted
I20260807 00:04:37.243000 21647 ts_tablet_manager.cc:1918] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:04:37.243136 21647 log.cc:1196] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/5a31d87ef9724fc9b77ed174b684ee9b
I20260807 00:04:37.243518 21647 ts_tablet_manager.cc:1939] T 5a31d87ef9724fc9b77ed174b684ee9b P b821bd8f67d143b480be20f116491df0: Deleting consensus metadata
I20260807 00:04:37.245033 20795 catalog_manager.cc:4841] TS b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351): tablet 5a31d87ef9724fc9b77ed174b684ee9b (table test-table [id=51256bffdd3e47068ee490b2ae30255f]) successfully deleted
W20260807 00:04:37.246348 20796 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:41535 (error 108)
I20260807 00:04:37.246595 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 21159
I20260807 00:04:37.250057 21608 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.249726 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41535 (local address 127.0.0.1:54822)
0807 00:04:37.249875 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.249956 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 00:04:37.250278 20796 catalog_manager.cc:4584] TS 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 5a31d87ef9724fc9b77ed174b684ee9b: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41535: connect: Connection refused (error 111)
I20260807 00:04:37.255872 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 21372
I20260807 00:04:37.270339 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 20771
I20260807 00:04:37.291420 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41003
--webserver_interface=127.30.204.126
--webserver_port=43971
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:41003
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:37.411182 21653 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:37.411504 21653 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:37.411566 21653 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:37.415762 21653 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:37.415843 21653 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:37.415861 21653 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:37.415877 21653 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:37.415892 21653 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:37.421590 21653 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:41003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=43971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21653
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:37.422770 21653 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:37.423882 21653 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:37.441589 21666 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:37.441589 21665 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:37.441680 21653 server_base.cc:1034] running on GCE node
W20260807 00:04:37.441596 21668 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:37.442476 21653 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:37.443934 21653 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:37.445135 21653 hybrid_clock.cc:648] HybridClock initialized: now 1786061077445109 us; error 42 us; skew 500 ppm
Aug 07 00:04:37 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:37.448354 21653 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:37.449517 21653 webserver.cc:460] Webserver started at http://127.30.204.126:43971/ using document root <none> and password file <none>
I20260807 00:04:37.450196 21653 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:37.450313 21653 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:37.454885 21653 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:04:37.457429 21683 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:37.458946 21653 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:04:37.459116 21653 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "086ddf14dfc541cb8e374c89ad462290"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "e25253e62c3137b7864e39c1addde0c6"
server_key_iv: "b6663a42f127c9c6038b83fcd4b7c2f9"
server_key_version: "encryptionkey@0"
I20260807 00:04:37.459877 21653 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:37.482378 21653 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:37.486923 21653 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:37.487135 21653 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:37.495092 21653 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:41003
I20260807 00:04:37.495393 21767 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:41003 every 8 connection(s)
I20260807 00:04:37.496506 21653 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:37.499780 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 21653
I20260807 00:04:37.500239 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:41535
--local_ip_for_outbound_sockets=127.30.204.65
--tserver_master_addrs=127.30.204.126:41003
--webserver_port=35967
--webserver_interface=127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:04:37.501708 21768 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:04:37.504590 21768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Bootstrap starting.
I20260807 00:04:37.523125 21768 log.cc:824] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:37.532716 21768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:04:37.533129 21768 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Bootstrap complete.
I20260807 00:04:37.537323 21768 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:37.537772 21768 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ddf14dfc541cb8e374c89ad462290, State: Initialized, Role: FOLLOWER
I20260807 00:04:37.538272 21768 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:37.538399 21768 raft_consensus.cc:388] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:04:37.538479 21768 raft_consensus.cc:482] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:04:37.538579 21768 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:04:37.540763 21768 raft_consensus.cc:504] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:37.541116 21768 leader_election.cc:302] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 086ddf14dfc541cb8e374c89ad462290; no voters:
I20260807 00:04:37.541458 21768 leader_election.cc:288] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:04:37.541579 21772 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:04:37.541769 21772 raft_consensus.cc:686] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 LEADER]: Becoming Leader. State: Replica: 086ddf14dfc541cb8e374c89ad462290, State: Running, Role: LEADER
I20260807 00:04:37.542083 21772 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:37.542879 21768 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:37.543727 21776 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086ddf14dfc541cb8e374c89ad462290. Latest consensus state: current_term: 2 leader_uuid: "086ddf14dfc541cb8e374c89ad462290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } } }
I20260807 00:04:37.543777 21774 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "086ddf14dfc541cb8e374c89ad462290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } } }
I20260807 00:04:37.543864 21776 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:37.543889 21774 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:37.548079 21800 catalog_manager.cc:1259] Loaded cluster ID: 74f02c21bac1462c80b5caf644a1b80c
I20260807 00:04:37.548149 21800 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: loading cluster ID for follower catalog manager: success
W20260807 00:04:37.548986 21800 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: acquiring CA information for follower catalog manager: Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG
W20260807 00:04:37.549054 21800 catalog_manager.cc:873] Runtime error: could not decrypt private key with the password returned by the configured command: unable to load data from memory: error reading private key: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D07803A:asn1 encoding routines:ASN1_ITEM_EX_D2I:nested asn1 error:tasn_dec.c:386:Type=X509_ALGOR error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=algor, Type=X509_SIG: failed to prepare follower catalog manager, will retry
I20260807 00:04:37.549122 21792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:37.552137 21792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:37.552656 21792 catalog_manager.cc:1259] Loaded cluster ID: 74f02c21bac1462c80b5caf644a1b80c
I20260807 00:04:37.552727 21792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:04:37.553743 21792 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260807 00:04:37.558171 21792 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260807 00:04:37.564512 21792 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:04:37.565500 21792 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Loaded TSK: 0
I20260807 00:04:37.566836 21792 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:04:37.631341 21770 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:37.631695 21770 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:37.631762 21770 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:37.635982 21770 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:37.636062 21770 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:37.636126 21770 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:04:37.641965 21770 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:41535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=35967
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21770
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:37.643236 21770 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:37.644469 21770 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:37.654899 21812 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:37.655074 21815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:37.654899 21811 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:37.655475 21770 server_base.cc:1034] running on GCE node
I20260807 00:04:37.655936 21770 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:37.656646 21770 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:37.657819 21770 hybrid_clock.cc:648] HybridClock initialized: now 1786061077657781 us; error 57 us; skew 500 ppm
Aug 07 00:04:37 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:37.660962 21770 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:04:37.662317 21770 webserver.cc:460] Webserver started at http://127.30.204.65:35967/ using document root <none> and password file <none>
I20260807 00:04:37.663130 21770 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:37.663354 21770 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:37.669097 21770 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 00:04:37.672027 21827 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:37.673621 21770 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:04:37.673770 21770 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "5391577b946d404db61ea40c0b93367e"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "6568aa1dc3fe4e072e86c05bfbc1470e"
server_key_iv: "4e48bca151840fed644bb5f35d2fe54a"
server_key_version: "encryptionkey@0"
I20260807 00:04:37.674507 21770 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:37.708420 21770 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:37.711550 21770 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:37.711719 21770 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:37.712321 21770 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:37.713428 21837 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:04:37.717798 21770 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:04:37.717888 21770 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260807 00:04:37.717955 21770 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:04:37.719864 21770 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:04:37.719918 21770 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.002s sys 0.000s
I20260807 00:04:37.720248 21837 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Bootstrap starting.
I20260807 00:04:37.729839 21770 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:41535
I20260807 00:04:37.729971 21999 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:41535 every 8 connection(s)
I20260807 00:04:37.730943 21770 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:04:37.739841 21837 log.cc:824] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:37.739996 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 21770
I20260807 00:04:37.740449 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:35349
--local_ip_for_outbound_sockets=127.30.204.66
--tserver_master_addrs=127.30.204.126:41003
--webserver_port=46701
--webserver_interface=127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:04:37 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:37.748999 22003 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.733270 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:46395)
0807 00:04:37.733864 (+ 594us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.733911 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.734755 (+ 844us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.736027 (+ 1272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.736044 (+ 17us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.736493 (+ 449us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:37.736982 (+ 489us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.736998 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.738382 (+ 1384us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.738385 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.739445 (+ 1060us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.739491 (+ 46us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.740595 (+ 1104us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.740599 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.740794 (+ 195us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.741375 (+ 581us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:37.741397 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:37.743192 (+ 1795us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:37.745794 (+ 2602us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:37.745799 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:37.745810 (+ 11us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:37.746086 (+ 276us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:37.746504 (+ 418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:37.746506 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:37.746508 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:37.746683 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:37.747427 (+ 744us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:37.747434 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:37.747754 (+ 320us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.747923 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.748214 (+ 291us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"thread_start_us":187,"threads_started":1}
I20260807 00:04:37.749225 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.733244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:46395 (local address 127.30.204.126:41003)
0807 00:04:37.733685 (+ 441us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.733710 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.734045 (+ 335us) server_negotiation.cc:407] Connection header received
0807 00:04:37.735202 (+ 1157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.735213 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.735605 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.735936 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:37.737167 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.738215 (+ 1048us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.739624 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.740443 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.740615 (+ 172us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.743431 (+ 2816us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:37.743438 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:37.743484 (+ 46us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:37.743523 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:37.745639 (+ 2116us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.746215 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.746221 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.746228 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.746399 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.746786 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.746790 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.746791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.747280 (+ 489us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:37.747421 (+ 141us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.748072 (+ 651us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.748389 (+ 317us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":250,"thread_start_us":131,"threads_started":1}
I20260807 00:04:37.749419 21837 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:04:37.749817 21837 tablet_bootstrap.cc:492] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Bootstrap complete.
I20260807 00:04:37.751053 21837 ts_tablet_manager.cc:1397] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Time spent bootstrapping tablet: real 0.031s user 0.024s sys 0.005s
I20260807 00:04:37.751854 22000 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41003
I20260807 00:04:37.752108 22000 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:37.752722 22000 heartbeater.cc:511] Master 127.30.204.126:41003 requested a full tablet report, sending...
I20260807 00:04:37.754037 21837 raft_consensus.cc:348] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.754724 21837 raft_consensus.cc:729] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5391577b946d404db61ea40c0b93367e, State: Initialized, Role: FOLLOWER
I20260807 00:04:37.755298 21837 consensus_queue.cc:260] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } } peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } }
I20260807 00:04:37.756102 21708 ts_manager.cc:194] Registered new tserver with Master: 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535)
I20260807 00:04:37.756554 21837 ts_tablet_manager.cc:1428] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
I20260807 00:04:37.759958 21708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:46395
I20260807 00:04:37.762903 22000 heartbeater.cc:503] Master 127.30.204.126:41003 was elected leader, sending a full tablet report...
Aug 07 00:04:37 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:04:37.772651 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.759354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41535 (local address 127.0.0.1:54834)
0807 00:04:37.760150 (+ 796us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.760212 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.760372 (+ 160us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.761126 (+ 754us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.761131 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.761622 (+ 491us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:37.761782 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.761792 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.763549 (+ 1757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.763552 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.764298 (+ 746us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.764312 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.765215 (+ 903us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.765220 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.765384 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.766052 (+ 668us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:37.766073 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:37.767934 (+ 1861us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:37.770561 (+ 2627us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:37.770574 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:37.770581 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:37.770910 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:37.771303 (+ 393us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:37.771305 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:37.771306 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:37.771384 (+ 78us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:37.772042 (+ 658us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:37.772048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:37.772178 (+ 130us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.772302 (+ 124us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.772500 (+ 198us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":504,"thread_start_us":150,"threads_started":1}
I20260807 00:04:37.772856 22010 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.759490 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54834 (local address 127.30.204.65:41535)
0807 00:04:37.760254 (+ 764us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.760267 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.760298 (+ 31us) server_negotiation.cc:407] Connection header received
0807 00:04:37.760559 (+ 261us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.760566 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.760814 (+ 248us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.760952 (+ 138us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:37.761928 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.763325 (+ 1397us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.764443 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.765028 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.765151 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.768147 (+ 2996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:37.768165 (+ 18us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:37.768185 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:37.768223 (+ 38us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:37.770380 (+ 2157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.771075 (+ 695us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.771083 (+ 8us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.771085 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.771208 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.771525 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.771528 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.771530 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.771914 (+ 384us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:37.772087 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.772447 (+ 360us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.772662 (+ 215us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":657,"thread_start_us":117,"threads_started":1}
I20260807 00:04:37.774091 21897 tablet_service.cc:1514] Processing DeleteTablet for tablet 5a31d87ef9724fc9b77ed174b684ee9b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:37 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:54834
I20260807 00:04:37.774994 22012 tablet_replica.cc:330] stopping tablet replica
I20260807 00:04:37.775315 22012 raft_consensus.cc:2229] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:04:37.775604 22012 raft_consensus.cc:2258] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:37.776706 22012 ts_tablet_manager.cc:1905] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:04:37.780820 22012 ts_tablet_manager.cc:1918] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:04:37.780925 22012 log.cc:1196] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/5a31d87ef9724fc9b77ed174b684ee9b
I20260807 00:04:37.781169 22012 ts_tablet_manager.cc:1939] T 5a31d87ef9724fc9b77ed174b684ee9b P 5391577b946d404db61ea40c0b93367e: Deleting consensus metadata
I20260807 00:04:37.783031 21689 catalog_manager.cc:4841] TS 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535): tablet 5a31d87ef9724fc9b77ed174b684ee9b (table test-table [id=51256bffdd3e47068ee490b2ae30255f]) successfully deleted
W20260807 00:04:37.874087 22006 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:37.874428 22006 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:37.874500 22006 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:37.878787 22006 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:37.878899 22006 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:37.879004 22006 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:04:37.884984 22006 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:35349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=46701
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22006
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:37.886209 22006 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:37.887342 22006 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:37.898451 22019 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:37.898618 22006 server_base.cc:1034] running on GCE node
W20260807 00:04:37.898654 22022 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:37.898470 22024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:37.899204 22006 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:37.899960 22006 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:37.901166 22006 hybrid_clock.cc:648] HybridClock initialized: now 1786061077901135 us; error 40 us; skew 500 ppm
Aug 07 00:04:37 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:37.904141 22006 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:04:37.905145 22006 webserver.cc:460] Webserver started at http://127.30.204.66:46701/ using document root <none> and password file <none>
I20260807 00:04:37.905741 22006 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:37.905823 22006 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:37.909996 22006 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:04:37.912406 22034 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:37.913779 22006 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:04:37.913892 22006 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "c1e17142eb5f4939a2ea75a920fbbe6a"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "ebf4c6cf7a966b6d1aec0e62f9facd61"
server_key_iv: "7799eebccddfc6d2bd91a5cf52bbec58"
server_key_version: "encryptionkey@0"
I20260807 00:04:37.914453 22006 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:37.937744 22006 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:37.941180 22006 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:37.941371 22006 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:37.941980 22006 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:37.942868 22006 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:37.942919 22006 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:37.942981 22006 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:37.942999 22006 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:37.952346 22006 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:35349
I20260807 00:04:37.952380 22211 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:35349 every 8 connection(s)
I20260807 00:04:37.953536 22006 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:04:37.956493 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 22006
I20260807 00:04:37.956993 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:43351
--local_ip_for_outbound_sockets=127.30.204.67
--tserver_master_addrs=127.30.204.126:41003
--webserver_port=43323
--webserver_interface=127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:04:37 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061077, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:37.970057 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.955624 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:55235 (local address 127.30.204.126:41003)
0807 00:04:37.955768 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:37.955772 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:37.956950 (+ 1178us) server_negotiation.cc:407] Connection header received
0807 00:04:37.958486 (+ 1536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:37.958490 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:37.958565 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:37.958647 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:37.960176 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.960959 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.962326 (+ 1367us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.962703 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.962827 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.965909 (+ 3082us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:37.965914 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:37.965917 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:37.965954 (+ 37us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:37.967671 (+ 1717us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.968242 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.968246 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.968248 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.968354 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:37.968789 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:37.968793 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:37.968795 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:37.969062 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:37.969175 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:37.969722 (+ 547us) server_negotiation.cc:299] Negotiation successful
0807 00:04:37.969875 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:04:37.970932 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:37.956100 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:55235)
0807 00:04:37.956746 (+ 646us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:37.956829 (+ 83us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:37.958244 (+ 1415us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:37.958864 (+ 620us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:37.958872 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:37.959273 (+ 401us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:37.959979 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.960002 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.961136 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.961139 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.962168 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:37.962183 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:37.962908 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:37.962912 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:37.963081 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:37.963824 (+ 743us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:37.963850 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:37.965671 (+ 1821us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:37.967804 (+ 2133us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:37.967811 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:37.967827 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:37.968117 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:37.968483 (+ 366us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:37.968486 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:37.968488 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:37.968671 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:37.969175 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:37.969180 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:37.969463 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 00:04:37.969627 (+ 164us) client_negotiation.cc:240] Negotiation successful
0807 00:04:37.969937 (+ 310us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":389,"thread_start_us":173,"threads_started":1}
I20260807 00:04:37.972183 22213 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41003
I20260807 00:04:37.972441 22213 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:37.973001 22213 heartbeater.cc:511] Master 127.30.204.126:41003 requested a full tablet report, sending...
I20260807 00:04:37.974115 21708 ts_manager.cc:194] Registered new tserver with Master: c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349)
I20260807 00:04:37.974725 21708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:55235
W20260807 00:04:38.080077 22217 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:38.080412 22217 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:38.080470 22217 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:38.085081 22217 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:38.085179 22217 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:38.085247 22217 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:04:38.091260 22217 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:43351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=43323
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22217
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:38.092521 22217 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:38.093644 22217 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:38.104144 22229 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:38.104226 22230 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:38.104226 22234 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:38.104517 22217 server_base.cc:1034] running on GCE node
I20260807 00:04:38.105043 22217 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:38.105829 22217 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:38.107033 22217 hybrid_clock.cc:648] HybridClock initialized: now 1786061078107003 us; error 44 us; skew 500 ppm
Aug 07 00:04:38 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:38.110296 22217 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:04:38.111361 22217 webserver.cc:460] Webserver started at http://127.30.204.67:43323/ using document root <none> and password file <none>
I20260807 00:04:38.111980 22217 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:38.112066 22217 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:38.116413 22217 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:04:38.118796 22243 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:38.120209 22217 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:04:38.120344 22217 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "b821bd8f67d143b480be20f116491df0"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "f454ad88d2da9d5bfd2e20a34787894c"
server_key_iv: "dbe198cce6c5d3485b8a41c89b638040"
server_key_version: "encryptionkey@0"
I20260807 00:04:38.120927 22217 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:38.153836 22217 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:38.157487 22217 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:38.157693 22217 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:38.158397 22217 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:38.159337 22217 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:38.159391 22217 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:38.159448 22217 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:38.159507 22217 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:38.168665 22217 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:43351
I20260807 00:04:38.168689 22412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:43351 every 8 connection(s)
I20260807 00:04:38.169751 22217 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:04:38.176329 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 22217
Aug 07 00:04:38 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:38.185676 22002 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.171750 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:56011 (local address 127.30.204.126:41003)
0807 00:04:38.171902 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.171907 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.172925 (+ 1018us) server_negotiation.cc:407] Connection header received
0807 00:04:38.174310 (+ 1385us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.174313 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.174366 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.174467 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:38.175813 (+ 1346us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.176350 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.177987 (+ 1637us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.178214 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.178296 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.181580 (+ 3284us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:38.181584 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:38.181586 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:38.181618 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:38.182991 (+ 1373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:38.183638 (+ 647us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:38.183641 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:38.183642 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:38.183739 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:38.184182 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:38.184185 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:38.184187 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:38.184451 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:38.184596 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.185313 (+ 717us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.185466 (+ 153us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:04:38.185856 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.177092 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40546 (local address 127.30.204.126:41003)
0807 00:04:38.177394 (+ 302us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.177398 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.177511 (+ 113us) server_negotiation.cc:407] Connection header received
0807 00:04:38.177635 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.177637 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.177681 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.177780 (+ 99us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:38.178677 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.179369 (+ 692us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.180552 (+ 1183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.180849 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.180959 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.182559 (+ 1600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:38.182563 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:38.182566 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:38.182605 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:38.184145 (+ 1540us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:38.184621 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:38.184624 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:38.184626 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:38.184712 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:38.184946 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:38.184949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:38.184951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:38.185169 (+ 218us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:38.185373 (+ 204us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.185558 (+ 185us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.185675 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":224,"thread_start_us":85,"threads_started":1}
I20260807 00:04:38.186730 22415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.172247 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:56011)
0807 00:04:38.172761 (+ 514us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:38.172808 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:38.174074 (+ 1266us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:38.174658 (+ 584us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:38.174667 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:38.175065 (+ 398us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:38.175626 (+ 561us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.175644 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.176516 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.176520 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.177863 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.177878 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.178382 (+ 504us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.178388 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.178648 (+ 260us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.179349 (+ 701us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:38.179367 (+ 18us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:38.181351 (+ 1984us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:38.183111 (+ 1760us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:38.183119 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:38.183143 (+ 24us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:38.183535 (+ 392us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:38.183826 (+ 291us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:38.183830 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:38.183832 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:38.184041 (+ 209us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:38.184606 (+ 565us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:38.184613 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:38.184996 (+ 383us) client_negotiation.cc:769] Sending connection context
0807 00:04:38.185235 (+ 239us) client_negotiation.cc:240] Negotiation successful
0807 00:04:38.185661 (+ 426us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":307,"thread_start_us":125,"threads_started":1}
I20260807 00:04:38.187923 22413 heartbeater.cc:347] Connected to a master server at 127.30.204.126:41003
I20260807 00:04:38.188170 22413 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:38.188681 22413 heartbeater.cc:511] Master 127.30.204.126:41003 requested a full tablet report, sending...
I20260807 00:04:38.189659 21708 ts_manager.cc:194] Registered new tserver with Master: b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351)
I20260807 00:04:38.190209 21708 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:56011
I20260807 00:04:38.199965 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:04:38.204859 22417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.200637 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40558 (local address 127.30.204.126:41003)
0807 00:04:38.200762 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.200766 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.200978 (+ 212us) server_negotiation.cc:407] Connection header received
0807 00:04:38.201142 (+ 164us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.201144 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.201256 (+ 112us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.201345 (+ 89us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:04:38.201557 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.202081 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.202961 (+ 880us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.203215 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.203309 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.203826 (+ 517us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:04:38.204328 (+ 502us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:04:38.204430 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.204654 (+ 224us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.204732 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260807 00:04:38.205623 21708 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:40558:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:04:38.209069 21708 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:04:38.217609 21897 tablet_service.cc:1467] Processing CreateTablet for tablet 9df72b8498214f3bb05a3bd1755b361b (DEFAULT_TABLE table=test-table [id=e5dd9c53b75d4e2da854d94c3cb7e48f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:38.218369 21897 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df72b8498214f3bb05a3bd1755b361b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:38.221185 21837 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Bootstrap starting.
I20260807 00:04:38.221989 22008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.216273 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35349 (local address 127.0.0.1:48316)
0807 00:04:38.216902 (+ 629us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:38.216918 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:38.217002 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:38.217540 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:38.217543 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:38.217562 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:38.217699 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.217706 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.219100 (+ 1394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.219104 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.220372 (+ 1268us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.220383 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.221522 (+ 1139us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.221525 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.221630 (+ 105us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.221705 (+ 75us) client_negotiation.cc:769] Sending connection context
0807 00:04:38.221764 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:04:38.221855 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":560}
I20260807 00:04:38.222141 21837 tablet_bootstrap.cc:654] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:38.222605 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.216596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:43351 (local address 127.0.0.1:52090)
0807 00:04:38.217083 (+ 487us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:38.217099 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:38.217205 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:38.218176 (+ 971us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:38.218179 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:38.218198 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:38.218432 (+ 234us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.218439 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.219995 (+ 1556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.219998 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.220922 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.220933 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.221983 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.221986 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.222307 (+ 321us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.222328 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 00:04:38.222396 (+ 68us) client_negotiation.cc:240] Negotiation successful
0807 00:04:38.222472 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":418,"thread_start_us":120,"threads_started":1}
I20260807 00:04:38.222769 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.216614 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.30.204.66:35349)
0807 00:04:38.216997 (+ 383us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.217011 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.217036 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:04:38.217121 (+ 85us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.217127 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.217376 (+ 249us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.217550 (+ 174us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:38.217848 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.218962 (+ 1114us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.220520 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.221378 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.222296 (+ 918us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.222467 (+ 171us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.222523 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.222615 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":272,"thread_start_us":143,"threads_started":1}
I20260807 00:04:38.222997 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.216968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52090 (local address 127.30.204.67:43351)
0807 00:04:38.217439 (+ 471us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.217448 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.217468 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:04:38.217546 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.217552 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.217765 (+ 213us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.217931 (+ 166us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:38.218677 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.219845 (+ 1168us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.221062 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.221877 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.221979 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.222153 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.222459 (+ 306us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.222569 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":147,"threads_started":1}
I20260807 00:04:38.224030 21837 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: No bootstrap required, opened a new log
I20260807 00:04:38.224117 21837 ts_tablet_manager.cc:1397] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260807 00:04:38.224587 21837 raft_consensus.cc:348] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.224706 21837 raft_consensus.cc:374] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:38.224745 21837 raft_consensus.cc:729] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5391577b946d404db61ea40c0b93367e, State: Initialized, Role: FOLLOWER
I20260807 00:04:38.224870 21837 consensus_queue.cc:260] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.225155 21837 ts_tablet_manager.cc:1428] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260807 00:04:38.226025 22109 tablet_service.cc:1467] Processing CreateTablet for tablet 9df72b8498214f3bb05a3bd1755b361b (DEFAULT_TABLE table=test-table [id=e5dd9c53b75d4e2da854d94c3cb7e48f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:38.226027 22317 tablet_service.cc:1467] Processing CreateTablet for tablet 9df72b8498214f3bb05a3bd1755b361b (DEFAULT_TABLE table=test-table [id=e5dd9c53b75d4e2da854d94c3cb7e48f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:38.227267 22317 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df72b8498214f3bb05a3bd1755b361b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:38.227258 22109 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 9df72b8498214f3bb05a3bd1755b361b. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 00:04:38.232218 22001 tablet.cc:2369] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:04:38.236414 22434 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Bootstrap starting.
I20260807 00:04:38.236974 22433 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Bootstrap starting.
I20260807 00:04:38.239627 22434 tablet_bootstrap.cc:654] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:38.239650 22433 tablet_bootstrap.cc:654] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:38.240623 22433 log.cc:824] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:38.240623 22434 log.cc:824] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:38.243120 22434 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: No bootstrap required, opened a new log
I20260807 00:04:38.243120 22433 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: No bootstrap required, opened a new log
I20260807 00:04:38.243360 22433 ts_tablet_manager.cc:1397] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Time spent bootstrapping tablet: real 0.007s user 0.000s sys 0.004s
I20260807 00:04:38.243360 22434 ts_tablet_manager.cc:1397] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:04:38.247669 22433 raft_consensus.cc:348] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.247669 22434 raft_consensus.cc:348] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.247999 22434 raft_consensus.cc:374] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:38.247998 22433 raft_consensus.cc:374] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:38.248063 22434 raft_consensus.cc:729] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c1e17142eb5f4939a2ea75a920fbbe6a, State: Initialized, Role: FOLLOWER
I20260807 00:04:38.248063 22433 raft_consensus.cc:729] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b821bd8f67d143b480be20f116491df0, State: Initialized, Role: FOLLOWER
I20260807 00:04:38.248595 22433 consensus_queue.cc:260] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.248595 22434 consensus_queue.cc:260] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.249399 22213 heartbeater.cc:503] Master 127.30.204.126:41003 was elected leader, sending a full tablet report...
I20260807 00:04:38.249399 22413 heartbeater.cc:503] Master 127.30.204.126:41003 was elected leader, sending a full tablet report...
I20260807 00:04:38.249794 22433 ts_tablet_manager.cc:1428] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:04:38.249892 22434 ts_tablet_manager.cc:1428] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
I20260807 00:04:38.378412 22439 raft_consensus.cc:482] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:38.378646 22439 raft_consensus.cc:504] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.380004 22439 leader_election.cc:288] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351), 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535)
I20260807 00:04:38.385536 22441 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.380431 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:36363 (local address 127.30.204.65:41535)
0807 00:04:38.380756 (+ 325us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.380761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.380783 (+ 22us) server_negotiation.cc:407] Connection header received
0807 00:04:38.380835 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.380838 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.380899 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.381024 (+ 125us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:38.381519 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.382350 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.384228 (+ 1878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.384958 (+ 730us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.385067 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.385206 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.385305 (+ 99us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.385387 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":215,"thread_start_us":82,"threads_started":1}
I20260807 00:04:38.385535 22430 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.380430 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:39283 (local address 127.30.204.67:43351)
0807 00:04:38.380558 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.380563 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.380845 (+ 282us) server_negotiation.cc:407] Connection header received
0807 00:04:38.380988 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.380992 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.381043 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.381116 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:38.381517 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.382349 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.384239 (+ 1890us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.384854 (+ 615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.385003 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.385064 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.385323 (+ 259us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.385402 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:04:38.385577 22215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.380348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41535 (local address 127.30.204.66:36363)
0807 00:04:38.380556 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:38.380573 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:38.380738 (+ 165us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:38.381087 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:38.381092 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:38.381120 (+ 28us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:38.381350 (+ 230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.381360 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.382506 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.382510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.384055 (+ 1545us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.384069 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.385071 (+ 1002us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.385075 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.385193 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.385211 (+ 18us) client_negotiation.cc:769] Sending connection context
0807 00:04:38.385277 (+ 66us) client_negotiation.cc:240] Negotiation successful
0807 00:04:38.385405 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"mutex_wait_us":116}
I20260807 00:04:38.385586 22444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.380351 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:43351 (local address 127.30.204.66:39283)
0807 00:04:38.380765 (+ 414us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:38.380777 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:38.380865 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:38.381163 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:38.381166 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:38.381181 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:38.381355 (+ 174us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.381361 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.382506 (+ 1145us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.382510 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.384060 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:38.384069 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.385016 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.385022 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:38.385161 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.385193 (+ 32us) client_negotiation.cc:769] Sending connection context
0807 00:04:38.385278 (+ 85us) client_negotiation.cc:240] Negotiation successful
0807 00:04:38.385405 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":354,"mutex_wait_us":266,"thread_start_us":130,"threads_started":1}
I20260807 00:04:38.386286 21936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df72b8498214f3bb05a3bd1755b361b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5391577b946d404db61ea40c0b93367e" is_pre_election: true
I20260807 00:04:38.386399 22348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df72b8498214f3bb05a3bd1755b361b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b821bd8f67d143b480be20f116491df0" is_pre_election: true
I20260807 00:04:38.386701 22348 raft_consensus.cc:2454] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 0.
I20260807 00:04:38.386703 21936 raft_consensus.cc:2454] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 0.
I20260807 00:04:38.387217 22037 leader_election.cc:302] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 5391577b946d404db61ea40c0b93367e, b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a; no voters:
I20260807 00:04:38.387531 22439 raft_consensus.cc:2781] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:04:38.387622 22439 raft_consensus.cc:482] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:04:38.387676 22439 raft_consensus.cc:3037] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:38.388782 22439 raft_consensus.cc:504] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.389142 22439 leader_election.cc:288] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 election: Requested vote from peers b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351), 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535)
I20260807 00:04:38.389551 22348 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df72b8498214f3bb05a3bd1755b361b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b821bd8f67d143b480be20f116491df0"
I20260807 00:04:38.389551 21936 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "9df72b8498214f3bb05a3bd1755b361b" candidate_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5391577b946d404db61ea40c0b93367e"
I20260807 00:04:38.389703 21936 raft_consensus.cc:3037] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:38.389705 22348 raft_consensus.cc:3037] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:38.391007 22348 raft_consensus.cc:2454] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 1.
I20260807 00:04:38.391007 21936 raft_consensus.cc:2454] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c1e17142eb5f4939a2ea75a920fbbe6a in term 1.
I20260807 00:04:38.391348 22038 leader_election.cc:302] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5391577b946d404db61ea40c0b93367e, c1e17142eb5f4939a2ea75a920fbbe6a; no voters:
I20260807 00:04:38.391535 22439 raft_consensus.cc:2781] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:38.391795 22439 raft_consensus.cc:686] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 LEADER]: Becoming Leader. State: Replica: c1e17142eb5f4939a2ea75a920fbbe6a, State: Running, Role: LEADER
I20260807 00:04:38.392168 22439 consensus_queue.cc:237] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.395483 21708 catalog_manager.cc:5495] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a reported cstate change: term changed from 0 to 1, leader changed from <none> to c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66). New cstate: current_term: 1 leader_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } health_report { overall_health: UNKNOWN } } }
I20260807 00:04:38.417953 22428 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.414543 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48332 (local address 127.30.204.66:35349)
0807 00:04:38.414675 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:38.414679 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:38.414733 (+ 54us) server_negotiation.cc:407] Connection header received
0807 00:04:38.414814 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:38.414816 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:38.414875 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:38.414960 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:04:38.415155 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.415713 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.416521 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:38.416757 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:38.416843 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:38.417091 (+ 248us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:04:38.417548 (+ 457us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:04:38.417635 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:38.417770 (+ 135us) server_negotiation.cc:299] Negotiation successful
0807 00:04:38.417825 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 00:04:38.420771 22414 tablet.cc:2369] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:04:38.423985 22348 raft_consensus.cc:1261] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Refusing update from remote peer c1e17142eb5f4939a2ea75a920fbbe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:38.423982 21936 raft_consensus.cc:1261] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Refusing update from remote peer c1e17142eb5f4939a2ea75a920fbbe6a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:38.424659 22439 consensus_queue.cc:1035] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:38.424883 22445 consensus_queue.cc:1035] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [LEADER]: Connected to new peer: Peer: permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:38.432813 22452 mvcc.cc:205] Tried to move back new op lower bound from 7315706177217343488 to 7315706177097261056. Current Snapshot: MvccSnapshot[applied={T|T < 7315706177217343488}]
I20260807 00:04:38.441129 21708 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:40558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:38.441311 21708 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:40558:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:38.443853 21708 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Sending DeleteTablet for 3 replicas of tablet 9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:38.444648 22109 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df72b8498214f3bb05a3bd1755b361b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:38 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:48316
I20260807 00:04:38.444726 22317 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df72b8498214f3bb05a3bd1755b361b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:38 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:52090
I20260807 00:04:38.444792 21897 tablet_service.cc:1514] Processing DeleteTablet for tablet 9df72b8498214f3bb05a3bd1755b361b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:38 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:54834
I20260807 00:04:38.445139 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 21770
I20260807 00:04:38.455531 22462 tablet_replica.cc:330] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: stopping tablet replica
I20260807 00:04:38.455971 22462 raft_consensus.cc:2229] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:04:38.456382 22462 raft_consensus.cc:2258] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:38.456600 22463 tablet_replica.cc:330] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: stopping tablet replica
I20260807 00:04:38.456983 22463 raft_consensus.cc:2229] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:04:38.457520 22463 raft_consensus.cc:2258] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:38.458282 22462 ts_tablet_manager.cc:1905] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:04:38.459192 22463 ts_tablet_manager.cc:1905] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:04:38.461259 22462 ts_tablet_manager.cc:1918] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:04:38.461390 22462 log.cc:1196] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:38.461768 22462 ts_tablet_manager.cc:1939] T 9df72b8498214f3bb05a3bd1755b361b P b821bd8f67d143b480be20f116491df0: Deleting consensus metadata
I20260807 00:04:38.462426 22463 ts_tablet_manager.cc:1918] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:04:38.462654 22463 log.cc:1196] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:38.463038 22463 ts_tablet_manager.cc:1939] T 9df72b8498214f3bb05a3bd1755b361b P c1e17142eb5f4939a2ea75a920fbbe6a: Deleting consensus metadata
W20260807 00:04:38.463065 21689 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:41535 (error 108)
I20260807 00:04:38.463297 21688 catalog_manager.cc:4841] TS b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351): tablet 9df72b8498214f3bb05a3bd1755b361b (table test-table [id=e5dd9c53b75d4e2da854d94c3cb7e48f]) successfully deleted
I20260807 00:04:38.464454 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 22006
I20260807 00:04:38.464460 21686 catalog_manager.cc:4841] TS c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349): tablet 9df72b8498214f3bb05a3bd1755b361b (table test-table [id=e5dd9c53b75d4e2da854d94c3cb7e48f]) successfully deleted
I20260807 00:04:38.464826 22426 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:38.464438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41535 (local address 127.0.0.1:54848)
0807 00:04:38.464647 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:38.464719 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41535: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119}
W20260807 00:04:38.474784 21689 catalog_manager.cc:4584] TS 5391577b946d404db61ea40c0b93367e (127.30.204.65:41535): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 9df72b8498214f3bb05a3bd1755b361b: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41535: connect: Connection refused (error 111)
I20260807 00:04:38.480492 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 22217
I20260807 00:04:38.513919 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 21653
I20260807 00:04:38.529798 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41003
--webserver_interface=127.30.204.126
--webserver_port=43971
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:41003
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:38.648177 22469 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:38.648510 22469 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:38.648571 22469 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:38.652825 22469 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:38.652911 22469 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:38.652938 22469 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:38.652961 22469 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:38.652980 22469 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:38.658692 22469 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:41003
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:41003
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=43971
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22469
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:38.659929 22469 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:38.661011 22469 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:04:38.670202 22469 server_base.cc:1034] running on GCE node
W20260807 00:04:38.670275 22474 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:38.670300 22478 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:38.670234 22481 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:38.671273 22469 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:38.672575 22469 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:38.673770 22469 hybrid_clock.cc:648] HybridClock initialized: now 1786061078673740 us; error 47 us; skew 500 ppm
Aug 07 00:04:38 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:38.677001 22469 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:38.678057 22469 webserver.cc:460] Webserver started at http://127.30.204.126:43971/ using document root <none> and password file <none>
I20260807 00:04:38.678665 22469 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:38.678736 22469 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:38.682977 22469 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:04:38.685467 22493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:38.686991 22469 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:04:38.687139 22469 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "086ddf14dfc541cb8e374c89ad462290"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "e25253e62c3137b7864e39c1addde0c6"
server_key_iv: "b6663a42f127c9c6038b83fcd4b7c2f9"
server_key_version: "encryptionkey@0"
I20260807 00:04:38.687812 22469 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:38.707681 22469 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:38.711424 22469 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:38.711683 22469 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:38.719353 22469 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:41003
I20260807 00:04:38.719668 22582 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:41003 every 8 connection(s)
I20260807 00:04:38.720619 22469 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:38.725507 22584 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:04:38.727543 22584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Bootstrap starting.
I20260807 00:04:38.727816 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 22469
I20260807 00:04:38.728264 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:41535
--local_ip_for_outbound_sockets=127.30.204.65
--tserver_master_addrs=127.30.204.126:41003
--webserver_port=35967
--webserver_interface=127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:04:38.751397 22584 log.cc:824] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:38.766781 22584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:04:38.767297 22584 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290: Bootstrap complete.
I20260807 00:04:38.772995 22584 raft_consensus.cc:348] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:38.773590 22584 raft_consensus.cc:729] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 086ddf14dfc541cb8e374c89ad462290, State: Initialized, Role: FOLLOWER
I20260807 00:04:38.774271 22584 consensus_queue.cc:260] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:38.774447 22584 raft_consensus.cc:388] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:04:38.774523 22584 raft_consensus.cc:482] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:04:38.774618 22584 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 2 FOLLOWER]: Advancing to term 3
I20260807 00:04:38.777582 22584 raft_consensus.cc:504] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:38.778047 22584 leader_election.cc:302] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 086ddf14dfc541cb8e374c89ad462290; no voters:
I20260807 00:04:38.778461 22584 leader_election.cc:288] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 00:04:38.778707 22590 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 00:04:38.778973 22590 raft_consensus.cc:686] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [term 3 LEADER]: Becoming Leader. State: Replica: 086ddf14dfc541cb8e374c89ad462290, State: Running, Role: LEADER
I20260807 00:04:38.779333 22590 consensus_queue.cc:237] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } }
I20260807 00:04:38.780040 22584 sys_catalog.cc:564] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:38.780964 22592 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 086ddf14dfc541cb8e374c89ad462290. Latest consensus state: current_term: 3 leader_uuid: "086ddf14dfc541cb8e374c89ad462290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } } }
I20260807 00:04:38.781096 22592 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:38.780932 22591 sys_catalog.cc:455] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "086ddf14dfc541cb8e374c89ad462290" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "086ddf14dfc541cb8e374c89ad462290" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 41003 } } }
I20260807 00:04:38.781224 22591 sys_catalog.cc:458] T 00000000000000000000000000000000 P 086ddf14dfc541cb8e374c89ad462290 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:38.781515 22604 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:38.787761 22604 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:38.788790 22604 catalog_manager.cc:1259] Loaded cluster ID: 74f02c21bac1462c80b5caf644a1b80c
I20260807 00:04:38.788878 22604 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260807 00:04:38.790155 22604 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f138a43c9ad google::LogMessage::Fail() at ??:0
@ 0x7f138a43f88a google::LogMessage::SendToLog() at ??:0
@ 0x7f138a43c4de google::LogMessage::Flush() at ??:0
@ 0x7f138a43da19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f1394d7ffdd _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f1394d80d2f kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f1394d7d579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f1394dc1640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f138c68e3d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f138c68ec83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f138c6903f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f138c67f512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f138dcc36db start_thread at ??:0
@ 0x7f1388f3a61f clone at ??:0
W20260807 00:04:38.879666 22588 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:38.879959 22588 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:38.880026 22588 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:38.884418 22588 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:38.884495 22588 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:38.884586 22588 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:04:38.890841 22588 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:41535
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=35967
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22588
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:38.892163 22588 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:38.893308 22588 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:38.903957 22641 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:38.903959 22640 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:38.904332 22643 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:38.904386 22588 server_base.cc:1034] running on GCE node
I20260807 00:04:38.904978 22588 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:38.906014 22588 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:38.907253 22588 hybrid_clock.cc:648] HybridClock initialized: now 1786061078907216 us; error 59 us; skew 500 ppm
Aug 07 00:04:38 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:38.911213 22588 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:04:38.912724 22588 webserver.cc:460] Webserver started at http://127.30.204.65:35967/ using document root <none> and password file <none>
I20260807 00:04:38.913496 22588 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:38.913599 22588 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:38.919529 22588 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:04:38.922669 22659 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:38.924441 22588 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:04:38.924614 22588 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "5391577b946d404db61ea40c0b93367e"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "6568aa1dc3fe4e072e86c05bfbc1470e"
server_key_iv: "4e48bca151840fed644bb5f35d2fe54a"
server_key_version: "encryptionkey@0"
I20260807 00:04:38.925419 22588 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:38.938831 22588 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:38.942411 22588 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:38.942643 22588 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:38.943321 22588 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:38.944617 22668 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:04:38.950573 22588 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:04:38.950685 22588 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.001s sys 0.000s
I20260807 00:04:38.950765 22588 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:04:38.952775 22588 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:04:38.952837 22588 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.001s sys 0.000s
I20260807 00:04:38.953204 22668 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Bootstrap starting.
I20260807 00:04:38.963446 22588 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:41535
I20260807 00:04:38.964181 22848 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:41535 every 8 connection(s)
I20260807 00:04:38.964798 22588 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:04:38.968030 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 22588
I20260807 00:04:38.968565 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:35349
--local_ip_for_outbound_sockets=127.30.204.66
--tserver_master_addrs=127.30.204.126:41003
--webserver_port=46701
--webserver_interface=127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:04:38.976807 22668 log.cc:824] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:38.987964 22668 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:04:38.988518 22668 tablet_bootstrap.cc:492] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Bootstrap complete.
I20260807 00:04:38.990013 22668 ts_tablet_manager.cc:1397] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Time spent bootstrapping tablet: real 0.037s user 0.031s sys 0.003s
I20260807 00:04:38.993709 22668 raft_consensus.cc:348] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.994362 22668 raft_consensus.cc:729] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5391577b946d404db61ea40c0b93367e, State: Initialized, Role: FOLLOWER
I20260807 00:04:38.995020 22668 consensus_queue.cc:260] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:38.996297 22668 ts_tablet_manager.cc:1428] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e: Time spent starting tablet: real 0.006s user 0.008s sys 0.000s
W20260807 00:04:39.043303 22852 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:04:38.967044 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:59823)
0807 00:04:38.968377 (+ 1333us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:38.968474 (+ 97us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:38.969842 (+ 1368us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:39.042244 (+ 72402us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":162,"threads_started":1}
W20260807 00:04:39.043886 22849 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:41003 (0 consecutive failures): Network error: Failed to ping master at 127.30.204.126:41003: Client connection negotiation failed: client connection to 127.30.204.126:41003: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:04:39.044674 22852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.044362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:55027)
0807 00:04:39.044502 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.044590 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":52}
I20260807 00:04:39.045295 22852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.045079 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:36119)
0807 00:04:39.045190 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.045241 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260807 00:04:39.045459 22849 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:04:39.094895 22853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:39.095243 22853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:39.095295 22853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:39.099906 22853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:39.099975 22853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:39.100039 22853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:04:39.105770 22853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:35349
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=46701
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:39.106995 22853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:39.108163 22853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:39.118594 22866 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:39.118615 22863 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:39.118613 22864 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:39.118903 22853 server_base.cc:1034] running on GCE node
I20260807 00:04:39.119422 22853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:39.120208 22853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:39.121404 22853 hybrid_clock.cc:648] HybridClock initialized: now 1786061079121369 us; error 49 us; skew 500 ppm
Aug 07 00:04:39 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:39.124365 22853 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:04:39.125540 22853 webserver.cc:460] Webserver started at http://127.30.204.66:46701/ using document root <none> and password file <none>
I20260807 00:04:39.126279 22853 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:39.126370 22853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:39.132021 22853 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:04:39.134526 22877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:39.136072 22853 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:04:39.136219 22853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "c1e17142eb5f4939a2ea75a920fbbe6a"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "ebf4c6cf7a966b6d1aec0e62f9facd61"
server_key_iv: "7799eebccddfc6d2bd91a5cf52bbec58"
server_key_version: "encryptionkey@0"
I20260807 00:04:39.136834 22853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:39.160420 22853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:39.164204 22853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:39.164412 22853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:39.165102 22853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:39.166132 22853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:39.166185 22853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:39.166232 22853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:39.166261 22853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:39.175882 22853 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:35349
I20260807 00:04:39.175904 23056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:35349 every 8 connection(s)
I20260807 00:04:39.177047 22853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:04:39.179282 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 22853
I20260807 00:04:39.179787 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:43351
--local_ip_for_outbound_sockets=127.30.204.67
--tserver_master_addrs=127.30.204.126:41003
--webserver_port=43323
--webserver_interface=127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:46873
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:04:39.181892 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.179528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:45533)
0807 00:04:39.180128 (+ 600us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.180744 (+ 616us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":326,"thread_start_us":116,"threads_started":1}
W20260807 00:04:39.182418 23058 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:41003 (0 consecutive failures): Network error: Failed to ping master at 127.30.204.126:41003: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
I20260807 00:04:39.183314 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.182944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:44355)
0807 00:04:39.183158 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.183243 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":75}
I20260807 00:04:39.184125 23063 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.183845 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:57865)
0807 00:04:39.183984 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.184040 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":60}
W20260807 00:04:39.184342 23058 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:04:39.302040 23064 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:39.302337 23064 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:39.302383 23064 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:39.306603 23064 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:39.306690 23064 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:39.306753 23064 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:04:39.312759 23064 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:46873
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:43351
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=43323
--tserver_master_addrs=127.30.204.126:41003
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23064
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:39.314025 23064 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:39.315305 23064 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:39.325829 23073 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:39.325837 23074 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:39.325842 23077 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:39.326361 23064 server_base.cc:1034] running on GCE node
I20260807 00:04:39.326849 23064 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:39.327744 23064 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:39.328923 23064 hybrid_clock.cc:648] HybridClock initialized: now 1786061079328907 us; error 33 us; skew 500 ppm
Aug 07 00:04:39 dist-test-slave-pww0 krb5kdc[20752](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061079, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:39.331969 23064 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:04:39.333019 23064 webserver.cc:460] Webserver started at http://127.30.204.67:43323/ using document root <none> and password file <none>
I20260807 00:04:39.333595 23064 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:39.333659 23064 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:39.337996 23064 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.000s sys 0.004s
I20260807 00:04:39.340576 23087 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:39.342435 23064 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:04:39.342586 23064 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "b821bd8f67d143b480be20f116491df0"
format_stamp: "Formatted at 2026-08-07 00:04:36 on dist-test-slave-pww0"
server_key: "f454ad88d2da9d5bfd2e20a34787894c"
server_key_iv: "dbe198cce6c5d3485b8a41c89b638040"
server_key_version: "encryptionkey@0"
I20260807 00:04:39.343209 23064 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:39.353755 23064 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:39.357365 23064 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:39.357545 23064 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:39.358152 23064 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:39.359035 23064 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:39.359083 23064 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:39.359126 23064 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:39.359146 23064 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:39.368604 23064 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:43351
I20260807 00:04:39.368640 23258 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:43351 every 8 connection(s)
I20260807 00:04:39.369768 23064 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:04:39.374061 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.372132 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:45883)
0807 00:04:39.372739 (+ 607us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.373228 (+ 489us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":371,"thread_start_us":145,"threads_started":1}
W20260807 00:04:39.374698 23259 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:41003 (0 consecutive failures): Network error: Failed to ping master at 127.30.204.126:41003: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
I20260807 00:04:39.375554 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.375213 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:53829)
0807 00:04:39.375368 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.375431 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":66}
I20260807 00:04:39.376338 23265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:39.376084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:50617)
0807 00:04:39.376217 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:39.376274 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
W20260807 00:04:39.376567 23259 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:04:39.379254 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 23064
I20260807 00:04:40.046751 23267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:40.046196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:58643)
0807 00:04:40.046547 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:40.046624 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":245,"thread_start_us":85,"threads_started":1}
I20260807 00:04:40.185513 23268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:40.184991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:52987)
0807 00:04:40.185313 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:40.185384 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":97,"threads_started":1}
I20260807 00:04:40.377724 23270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:40.377228 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:54679)
0807 00:04:40.377518 (+ 290us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:40.377586 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":85,"threads_started":1}
I20260807 00:04:40.852787 23273 raft_consensus.cc:482] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:40.853057 23273 raft_consensus.cc:504] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:40.854346 23273 leader_election.cc:288] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349), b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351)
Aug 07 00:04:40 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.66@KRBTEST.COM
Aug 07 00:04:40 dist-test-slave-pww0 krb5kdc[20752](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061078, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.67@KRBTEST.COM
I20260807 00:04:40.869076 23274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:40.854586 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:43351 (local address 127.30.204.65:41023)
0807 00:04:40.854883 (+ 297us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:40.854895 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:40.854984 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:40.856756 (+ 1772us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:40.856762 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:40.857347 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:40.857873 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:40.857890 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.859305 (+ 1415us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.859308 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:40.860275 (+ 967us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:40.860286 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.861003 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.861007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:40.861252 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:40.861910 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:40.861936 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:40.864140 (+ 2204us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:40.866347 (+ 2207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:40.866355 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:40.866371 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:40.866781 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:40.867152 (+ 371us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:40.867155 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:40.867157 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:40.867315 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:40.868023 (+ 708us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:40.868029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:40.868282 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 00:04:40.868451 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 00:04:40.868862 (+ 411us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":185,"spinlock_wait_cycles":38784,"thread_start_us":73,"threads_started":1}
I20260807 00:04:40.869091 23276 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:40.854737 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:34415 (local address 127.30.204.66:35349)
0807 00:04:40.855204 (+ 467us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:40.855220 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:40.855503 (+ 283us) server_negotiation.cc:407] Connection header received
0807 00:04:40.856238 (+ 735us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:40.856243 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:40.856537 (+ 294us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:40.856796 (+ 259us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:40.858045 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.859048 (+ 1003us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.860393 (+ 1345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.860878 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.861036 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:40.863974 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:40.863983 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:40.864003 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:40.864075 (+ 72us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:40.866300 (+ 2225us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:40.866909 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:40.866915 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:40.866920 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:40.867053 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:40.867409 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:40.867413 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:40.867414 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:40.867904 (+ 490us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:40.868031 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:40.868606 (+ 575us) server_negotiation.cc:299] Negotiation successful
0807 00:04:40.868889 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":343,"thread_start_us":107,"threads_started":1}
I20260807 00:04:40.869114 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:40.854590 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:35349 (local address 127.30.204.65:34415)
0807 00:04:40.855378 (+ 788us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:40.855389 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:40.855526 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:40.856762 (+ 1236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:40.856766 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:40.857347 (+ 581us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:40.857874 (+ 527us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:40.857890 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.859197 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.859200 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:40.860274 (+ 1074us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:40.860285 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.861003 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.861007 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:40.861250 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:40.861943 (+ 693us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:40.861955 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:40.863671 (+ 1716us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:40.866410 (+ 2739us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:40.866412 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:40.866414 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:40.866776 (+ 362us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:40.867148 (+ 372us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:40.867150 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:40.867152 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:40.867310 (+ 158us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:40.868023 (+ 713us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:40.868029 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:40.868282 (+ 253us) client_negotiation.cc:769] Sending connection context
0807 00:04:40.868450 (+ 168us) client_negotiation.cc:240] Negotiation successful
0807 00:04:40.868936 (+ 486us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":731,"mutex_wait_us":757,"spinlock_wait_cycles":29056,"thread_start_us":76,"threads_started":1}
I20260807 00:04:40.869102 23277 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:40.854802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:41023 (local address 127.30.204.67:43351)
0807 00:04:40.855205 (+ 403us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:40.855229 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:40.855264 (+ 35us) server_negotiation.cc:407] Connection header received
0807 00:04:40.855997 (+ 733us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:40.856005 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:40.856370 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:40.856673 (+ 303us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:40.858051 (+ 1378us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.859142 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.860393 (+ 1251us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:40.860878 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:40.861036 (+ 158us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:40.864401 (+ 3365us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:40.864407 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:40.864420 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:40.864454 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:40.866190 (+ 1736us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:40.866912 (+ 722us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:40.866918 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:40.866923 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:40.867054 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:40.867414 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:40.867418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:40.867419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:40.867904 (+ 485us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:40.868031 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:40.868610 (+ 579us) server_negotiation.cc:299] Negotiation successful
0807 00:04:40.868908 (+ 298us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":297,"thread_start_us":57,"threads_started":1}
W20260807 00:04:40.871965 22662 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
W20260807 00:04:40.872192 22661 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:40.872287 22661 leader_election.cc:302] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5391577b946d404db61ea40c0b93367e; no voters: b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a
I20260807 00:04:40.872607 23273 raft_consensus.cc:2726] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:04:41.048071 23279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:41.047745 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:49771)
0807 00:04:41.047897 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:41.047968 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260807 00:04:41.187048 23281 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:41.186437 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:36891)
0807 00:04:41.186825 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:41.186902 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":283,"thread_start_us":108,"threads_started":1}
I20260807 00:04:41.379122 23282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:41.378602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:52483)
0807 00:04:41.378931 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:41.378992 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":247,"thread_start_us":81,"threads_started":1}
I20260807 00:04:42.049305 23283 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:42.048876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:55529)
0807 00:04:42.049124 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:42.049199 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":79,"threads_started":1}
I20260807 00:04:42.188575 23290 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:42.188059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:34903)
0807 00:04:42.188392 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:42.188469 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":84,"threads_started":1}
I20260807 00:04:42.380577 23293 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:42.380043 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:60493)
0807 00:04:42.380399 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:42.380472 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":106,"threads_started":1}
I20260807 00:04:43.050827 23295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:43.050240 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:59103)
0807 00:04:43.050633 (+ 393us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:43.050713 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":95,"threads_started":1}
I20260807 00:04:43.090618 23296 raft_consensus.cc:482] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:43.090764 23296 raft_consensus.cc:504] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:43.091174 23296 leader_election.cc:288] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349), b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351)
W20260807 00:04:43.092257 22662 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
W20260807 00:04:43.092413 22661 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:43.092481 22661 leader_election.cc:302] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5391577b946d404db61ea40c0b93367e; no voters: b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a
I20260807 00:04:43.092681 23296 raft_consensus.cc:2726] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:04:43.190119 23297 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:43.189562 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:52835)
0807 00:04:43.189940 (+ 378us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:43.190014 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":142,"threads_started":1}
I20260807 00:04:43.381961 23298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:43.381484 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:49731)
0807 00:04:43.381790 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:43.381859 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":109,"threads_started":1}
I20260807 00:04:44.052556 23299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:44.051969 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:58545)
0807 00:04:44.052353 (+ 384us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:44.052432 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":302,"thread_start_us":97,"threads_started":1}
I20260807 00:04:44.191706 23300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:44.191058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:58743)
0807 00:04:44.191490 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:44.191572 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":299,"thread_start_us":88,"threads_started":1}
I20260807 00:04:44.383654 23302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:44.382919 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:50329)
0807 00:04:44.383289 (+ 370us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:44.383365 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":272,"thread_start_us":115,"threads_started":1}
I20260807 00:04:45.053953 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:45.053473 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:44071)
0807 00:04:45.053785 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:45.053853 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":89,"threads_started":1}
I20260807 00:04:45.193128 23306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:45.192615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:57061)
0807 00:04:45.192928 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:45.192999 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":82,"threads_started":1}
I20260807 00:04:45.365932 23308 raft_consensus.cc:482] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:45.366099 23308 raft_consensus.cc:504] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:45.366577 23308 leader_election.cc:288] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349), b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351)
W20260807 00:04:45.367479 22662 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
W20260807 00:04:45.367632 22661 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:45.367723 22661 leader_election.cc:302] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5391577b946d404db61ea40c0b93367e; no voters: b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a
I20260807 00:04:45.367883 23308 raft_consensus.cc:2726] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:04:45.385167 23309 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:45.384565 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:43567)
0807 00:04:45.384985 (+ 420us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:45.385060 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":219,"threads_started":1}
I20260807 00:04:46.055326 23310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:46.054841 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:53821)
0807 00:04:46.055137 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:46.055206 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":103,"threads_started":1}
I20260807 00:04:46.194693 23312 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:46.194056 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:38237)
0807 00:04:46.194480 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:46.194556 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":323,"thread_start_us":117,"threads_started":1}
I20260807 00:04:46.386680 23313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:46.386114 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:42175)
0807 00:04:46.386513 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:46.386572 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":316,"thread_start_us":99,"threads_started":1}
I20260807 00:04:47.056612 23315 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:47.056134 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:52257)
0807 00:04:47.056433 (+ 299us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:47.056500 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":100,"threads_started":1}
I20260807 00:04:47.196224 23316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:47.195718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:45577)
0807 00:04:47.196039 (+ 321us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:47.196115 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":228,"thread_start_us":105,"threads_started":1}
I20260807 00:04:47.388250 23318 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:47.387713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:40251)
0807 00:04:47.388041 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:47.388125 (+ 84us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":82,"threads_started":1}
I20260807 00:04:48.058081 23323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:48.057533 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:48397)
0807 00:04:48.057891 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:48.057962 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":275,"thread_start_us":103,"threads_started":1}
I20260807 00:04:48.197737 23325 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:48.197112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:59927)
0807 00:04:48.197507 (+ 395us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:48.197593 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":129,"threads_started":1}
I20260807 00:04:48.389745 23328 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:48.389204 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:56329)
0807 00:04:48.389568 (+ 364us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:48.389643 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":265,"thread_start_us":85,"threads_started":1}
I20260807 00:04:49.059836 23329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:49.059168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:35043)
0807 00:04:49.059609 (+ 441us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:49.059695 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":303,"thread_start_us":125,"threads_started":1}
I20260807 00:04:49.199213 23330 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:49.198714 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:44085)
0807 00:04:49.199025 (+ 311us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:49.199100 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":83,"threads_started":1}
I20260807 00:04:49.391238 23331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:49.390720 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:43279)
0807 00:04:49.391045 (+ 325us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:49.391119 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":104,"threads_started":1}
I20260807 00:04:50.061348 23332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:50.060852 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:47013)
0807 00:04:50.061184 (+ 332us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:50.061244 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":113,"threads_started":1}
I20260807 00:04:50.200565 23335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:50.200099 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:44887)
0807 00:04:50.200387 (+ 288us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:50.200458 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":202,"thread_start_us":83,"threads_started":1}
I20260807 00:04:50.327951 23336 raft_consensus.cc:482] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:50.328141 23336 raft_consensus.cc:504] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:50.328644 23336 leader_election.cc:288] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349), b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351)
W20260807 00:04:50.329648 22661 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
W20260807 00:04:50.329775 22662 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:50.329931 22662 leader_election.cc:302] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5391577b946d404db61ea40c0b93367e; no voters: b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a
I20260807 00:04:50.330080 23336 raft_consensus.cc:2726] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:04:50.392740 23337 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:50.392143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:33303)
0807 00:04:50.392545 (+ 402us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:50.392626 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":114,"threads_started":1}
I20260807 00:04:51.062736 23340 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:51.062191 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:46991)
0807 00:04:51.062541 (+ 350us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:51.062623 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":106,"threads_started":1}
I20260807 00:04:51.201799 23342 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:51.201364 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:49145)
0807 00:04:51.201624 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:51.201685 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260807 00:04:51.394183 23346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:51.393617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:34973)
0807 00:04:51.393986 (+ 369us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:51.394074 (+ 88us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":262,"thread_start_us":113,"threads_started":1}
I20260807 00:04:52.064218 23349 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:52.063711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:57511)
0807 00:04:52.064037 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:52.064111 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":89,"threads_started":1}
I20260807 00:04:52.203105 23350 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:52.202623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:60443)
0807 00:04:52.202919 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:52.202986 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":211,"thread_start_us":85,"threads_started":1}
I20260807 00:04:52.395597 23354 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:52.395045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:44443)
0807 00:04:52.395371 (+ 326us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:52.395444 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":103,"threads_started":1}
I20260807 00:04:53.065685 23355 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:53.065165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:40243)
0807 00:04:53.065480 (+ 315us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:53.065553 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":93,"threads_started":1}
I20260807 00:04:53.204502 23356 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:53.203993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:33359)
0807 00:04:53.204322 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:53.204395 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":234,"thread_start_us":89,"threads_started":1}
I20260807 00:04:53.286044 23357 raft_consensus.cc:482] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:53.286233 23357 raft_consensus.cc:504] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c1e17142eb5f4939a2ea75a920fbbe6a" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 35349 } } peers { permanent_uuid: "b821bd8f67d143b480be20f116491df0" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 43351 } } peers { permanent_uuid: "5391577b946d404db61ea40c0b93367e" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41535 } }
I20260807 00:04:53.286710 23357 leader_election.cc:288] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349), b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351)
W20260807 00:04:53.287565 22662 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer b821bd8f67d143b480be20f116491df0 (127.30.204.67:43351): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
W20260807 00:04:53.287720 22661 leader_election.cc:341] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer c1e17142eb5f4939a2ea75a920fbbe6a (127.30.204.66:35349): Not found: Tablet not found: 9df72b8498214f3bb05a3bd1755b361b
I20260807 00:04:53.287832 22661 leader_election.cc:302] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 5391577b946d404db61ea40c0b93367e; no voters: b821bd8f67d143b480be20f116491df0, c1e17142eb5f4939a2ea75a920fbbe6a
I20260807 00:04:53.288010 23357 raft_consensus.cc:2726] T 9df72b8498214f3bb05a3bd1755b361b P 5391577b946d404db61ea40c0b93367e [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:04:53.396886 23358 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:53.396442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:58611)
0807 00:04:53.396704 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:53.396786 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":82,"threads_started":1}
I20260807 00:04:54.067129 23359 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:54.066685 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.65:49063)
0807 00:04:54.066967 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:54.067027 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":115,"threads_started":1}
I20260807 00:04:54.206149 23361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:54.205573 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.66:40355)
0807 00:04:54.205958 (+ 385us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:54.206031 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":294,"thread_start_us":113,"threads_started":1}
I20260807 00:04:54.388506 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 22588
I20260807 00:04:54.399176 23363 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:54.397876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:41003 (local address 127.30.204.67:51663)
0807 00:04:54.398978 (+ 1102us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:54.399057 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:41003: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":998,"thread_start_us":806,"threads_started":1}
I20260807 00:04:54.404809 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 22853
I20260807 00:04:54.420691 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 23064
2026-08-07T00:04:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (19754 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:04:54 dist-test-slave-pww0 krb5kdc[23366](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:04:54 dist-test-slave-pww0 krb5kdc[23366](info): set up 2 sockets
Aug 07 00:04:54 dist-test-slave-pww0 krb5kdc[23366](info): commencing operation
krb5kdc: starting...
W20260807 00:04:55.596300 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:04:55 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:04:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:04:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:04:55.730589 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36415
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:36415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:55.847772 23386 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:55.848088 23386 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:55.848150 23386 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:55.852428 23386 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 00:04:55.852505 23386 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 00:04:55.852528 23386 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 00:04:55.852545 23386 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:04:55.852571 23386 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:55.852607 23386 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:55.852638 23386 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:04:55.852659 23386 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:04:55.858461 23386 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:36415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23386
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:55.859745 23386 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:55.860862 23386 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:55.869912 23394 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:55.869902 23396 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:55.869910 23393 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:55.870293 23386 server_base.cc:1034] running on GCE node
I20260807 00:04:55.870919 23386 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:55.872193 23386 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:55.873405 23386 hybrid_clock.cc:648] HybridClock initialized: now 1786061095873376 us; error 48 us; skew 500 ppm
Aug 07 00:04:55 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061095, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:55.876554 23386 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:04:55.877658 23386 webserver.cc:460] Webserver started at http://127.30.204.126:43239/ using document root <none> and password file <none>
I20260807 00:04:55.878276 23386 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:55.878345 23386 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:55.878530 23386 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:55.880857 23386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "a19425ff05a345668eb4b1729cffe2a0"
format_stamp: "Formatted at 2026-08-07 00:04:55 on dist-test-slave-pww0"
server_key: "6672a34c216625ef5484a239633aeaaf"
server_key_iv: "d44d9a3fdedca6c68b5390c2b13332b9"
server_key_version: "encryptionkey@0"
I20260807 00:04:55.881443 23386 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "a19425ff05a345668eb4b1729cffe2a0"
format_stamp: "Formatted at 2026-08-07 00:04:55 on dist-test-slave-pww0"
server_key: "6672a34c216625ef5484a239633aeaaf"
server_key_iv: "d44d9a3fdedca6c68b5390c2b13332b9"
server_key_version: "encryptionkey@0"
I20260807 00:04:55.885553 23386 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:04:55.888499 23413 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:55.889981 23386 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:04:55.890115 23386 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "a19425ff05a345668eb4b1729cffe2a0"
format_stamp: "Formatted at 2026-08-07 00:04:55 on dist-test-slave-pww0"
server_key: "6672a34c216625ef5484a239633aeaaf"
server_key_iv: "d44d9a3fdedca6c68b5390c2b13332b9"
server_key_version: "encryptionkey@0"
I20260807 00:04:55.890259 23386 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:55.904532 23386 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:55.908214 23386 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:55.908427 23386 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:55.915654 23386 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:36415
I20260807 00:04:55.915937 23501 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:36415 every 8 connection(s)
I20260807 00:04:55.916731 23386 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:04:55.918586 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 23386
I20260807 00:04:55.918699 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:04:55.918992 31537 external_mini_cluster.cc:1428] Setting key 4c5889660b4c0fc57eae88134910c085
I20260807 00:04:55.920235 23502 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:55.927811 23502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Bootstrap starting.
Aug 07 00:04:55 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061095, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:55.930828 23502 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:55.931938 23502 log.cc:824] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:55.934605 23502 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: No bootstrap required, opened a new log
I20260807 00:04:55.936203 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:55.920627 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60352 (local address 127.30.204.126:36415)
0807 00:04:55.921104 (+ 477us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:55.921125 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:55.921180 (+ 55us) server_negotiation.cc:407] Connection header received
0807 00:04:55.922092 (+ 912us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:55.922120 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:55.922578 (+ 458us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:55.922989 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:55.923822 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:55.925379 (+ 1557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:55.926452 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:55.927199 (+ 747us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:55.927380 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:55.930421 (+ 3041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:55.930429 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:55.930449 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:55.930494 (+ 45us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:55.932695 (+ 2201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:55.933277 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:55.933284 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:55.933291 (+ 7us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:55.933447 (+ 156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:55.933811 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:55.933815 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:55.933817 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:55.934386 (+ 569us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:55.934588 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:55.934969 (+ 381us) server_negotiation.cc:299] Negotiation successful
0807 00:04:55.935305 (+ 336us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":312,"thread_start_us":142,"threads_started":1}
I20260807 00:04:55.938625 23502 raft_consensus.cc:348] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:04:55.938966 23502 raft_consensus.cc:374] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:55.939036 23502 raft_consensus.cc:729] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19425ff05a345668eb4b1729cffe2a0, State: Initialized, Role: FOLLOWER
I20260807 00:04:55.939684 23502 consensus_queue.cc:260] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:04:55.939862 23502 raft_consensus.cc:388] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:04:55.939939 23502 raft_consensus.cc:482] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:04:55.940044 23502 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:55.941107 23502 raft_consensus.cc:504] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:04:55.941490 23502 leader_election.cc:302] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19425ff05a345668eb4b1729cffe2a0; no voters:
I20260807 00:04:55.941849 23502 leader_election.cc:288] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:04:55.942006 23509 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:55.942250 23509 raft_consensus.cc:686] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 LEADER]: Becoming Leader. State: Replica: a19425ff05a345668eb4b1729cffe2a0, State: Running, Role: LEADER
I20260807 00:04:55.942601 23509 consensus_queue.cc:237] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:04:55.943203 23502 sys_catalog.cc:564] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:04:55.944298 23513 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a19425ff05a345668eb4b1729cffe2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } } }
I20260807 00:04:55.944450 23513 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:55.944761 23516 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a19425ff05a345668eb4b1729cffe2a0. Latest consensus state: current_term: 1 leader_uuid: "a19425ff05a345668eb4b1729cffe2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } } }
I20260807 00:04:55.944922 23516 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:04:55.945379 23529 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:04:55.948551 23529 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:04:55.954906 23529 catalog_manager.cc:1347] Generated new cluster ID: fe43b79711d04b5c84ee192046b002e1
I20260807 00:04:55.955003 23529 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:04:55.978909 23529 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:04:55.980413 23529 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:04:55.987641 23529 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Generated new TSK 0
I20260807 00:04:55.988463 23529 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:04:56.026779 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:36415
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:56.149151 23548 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:56.149493 23548 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:56.149546 23548 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:56.153942 23548 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:56.154029 23548 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:56.154093 23548 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:04:56.159837 23548 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23548
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:56.161087 23548 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:56.162284 23548 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:56.172945 23556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:56.172950 23559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:56.173072 23548 server_base.cc:1034] running on GCE node
W20260807 00:04:56.172966 23557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:56.173785 23548 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:56.174506 23548 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:56.175678 23548 hybrid_clock.cc:648] HybridClock initialized: now 1786061096175643 us; error 47 us; skew 500 ppm
Aug 07 00:04:56 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061096, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:56.183401 23548 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:04:56.184558 23548 webserver.cc:460] Webserver started at http://127.30.204.65:43971/ using document root <none> and password file <none>
I20260807 00:04:56.185180 23548 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:56.185256 23548 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:56.185734 23548 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:56.188066 23548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "743a6966a0454730a1bce6148ab272ac"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "b6fea95a4cfdb65006cf0464370210dc"
server_key_iv: "53d4810c9cecb98383a0cedd1f38f50e"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.188675 23548 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "743a6966a0454730a1bce6148ab272ac"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "b6fea95a4cfdb65006cf0464370210dc"
server_key_iv: "53d4810c9cecb98383a0cedd1f38f50e"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.192782 23548 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.004s
I20260807 00:04:56.195511 23570 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.196951 23548 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:04:56.197089 23548 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "743a6966a0454730a1bce6148ab272ac"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "b6fea95a4cfdb65006cf0464370210dc"
server_key_iv: "53d4810c9cecb98383a0cedd1f38f50e"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.197217 23548 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:56.210453 23548 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:56.214007 23548 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:56.214213 23548 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:56.214874 23548 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:56.215845 23548 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:56.215894 23548 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.215941 23548 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:56.215969 23548 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.225178 23548 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:41603
I20260807 00:04:56.225205 23745 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:41603 every 8 connection(s)
I20260807 00:04:56.226184 23548 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:04:56.235440 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 23548
I20260807 00:04:56.235603 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:04:56.235889 31537 external_mini_cluster.cc:1428] Setting key 9cd4837066d79c7a2ce52e4e1d283af6
Aug 07 00:04:56 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061096, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:56.241830 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.228132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:37557 (local address 127.30.204.126:36415)
0807 00:04:56.228354 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:56.228359 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:56.229558 (+ 1199us) server_negotiation.cc:407] Connection header received
0807 00:04:56.230549 (+ 991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:56.230553 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:56.230621 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:56.230718 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:56.232148 (+ 1430us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.232722 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.233985 (+ 1263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.234250 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.234360 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.237584 (+ 3224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:56.237588 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:56.237590 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:56.237619 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:56.239356 (+ 1737us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.240029 (+ 673us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.240032 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.240033 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.240136 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.240533 (+ 397us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.240535 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.240537 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.240759 (+ 222us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:56.240853 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:56.241543 (+ 690us) server_negotiation.cc:299] Negotiation successful
0807 00:04:56.241693 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 00:04:56.243008 23748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.228566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:37557)
0807 00:04:56.229344 (+ 778us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:56.229396 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:56.230299 (+ 903us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:56.230925 (+ 626us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:56.230933 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:56.231355 (+ 422us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:56.231957 (+ 602us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.231974 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.232875 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.232878 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.233836 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.233848 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.234430 (+ 582us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.234435 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.234633 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.235495 (+ 862us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:56.235514 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:56.237400 (+ 1886us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:56.239569 (+ 2169us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:56.239577 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:56.239594 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:56.239904 (+ 310us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:56.240234 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:56.240244 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:56.240246 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:56.240423 (+ 177us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:56.240871 (+ 448us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:56.240880 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:56.241187 (+ 307us) client_negotiation.cc:769] Sending connection context
0807 00:04:56.241410 (+ 223us) client_negotiation.cc:240] Negotiation successful
0807 00:04:56.241799 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":125,"threads_started":1}
I20260807 00:04:56.244581 23746 heartbeater.cc:347] Connected to a master server at 127.30.204.126:36415
I20260807 00:04:56.244885 23746 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:56.245618 23746 heartbeater.cc:511] Master 127.30.204.126:36415 requested a full tablet report, sending...
I20260807 00:04:56.247594 23444 ts_manager.cc:194] Registered new tserver with Master: 743a6966a0454730a1bce6148ab272ac (127.30.204.65:41603)
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:04:56.248946 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:37557
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:04:56.267503 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:36415
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:56.386792 23753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:56.387099 23753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:56.387141 23753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:56.391419 23753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:56.391553 23753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:56.391620 23753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:04:56.397477 23753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:56.398648 23753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:56.399787 23753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:56.410180 23760 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:56.410151 23763 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:56.410151 23759 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:56.410542 23753 server_base.cc:1034] running on GCE node
I20260807 00:04:56.410991 23753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:56.411779 23753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:56.412947 23753 hybrid_clock.cc:648] HybridClock initialized: now 1786061096412924 us; error 37 us; skew 500 ppm
Aug 07 00:04:56 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061096, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:56.415948 23753 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:04:56.417023 23753 webserver.cc:460] Webserver started at http://127.30.204.66:34273/ using document root <none> and password file <none>
I20260807 00:04:56.417637 23753 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:56.417711 23753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:56.417939 23753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:56.420248 23753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "053baa0d2ba14e6baef7f84a8eb38106"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "25deb440e738f0fbc1b294dcf09ff545"
server_key_iv: "86907273a3f2b459bdc3fb87147f9ba7"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.420872 23753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "053baa0d2ba14e6baef7f84a8eb38106"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "25deb440e738f0fbc1b294dcf09ff545"
server_key_iv: "86907273a3f2b459bdc3fb87147f9ba7"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.424908 23753 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:04:56.427641 23774 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.428995 23753 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.001s
I20260807 00:04:56.429124 23753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "053baa0d2ba14e6baef7f84a8eb38106"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "25deb440e738f0fbc1b294dcf09ff545"
server_key_iv: "86907273a3f2b459bdc3fb87147f9ba7"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.429251 23753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:56.469025 23753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:56.472519 23753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:56.472733 23753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:56.473397 23753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:56.474327 23753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:56.474380 23753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.474439 23753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:56.474464 23753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.483762 23753 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:44529
I20260807 00:04:56.483794 23952 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:44529 every 8 connection(s)
I20260807 00:04:56.484760 23753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:04:56.486142 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 23753
I20260807 00:04:56.486276 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:04:56.486557 31537 external_mini_cluster.cc:1428] Setting key 0ff49e6acd12dad1eb98bef6dab5df6f
Aug 07 00:04:56 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061096, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
I20260807 00:04:56.500876 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.486943 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:51843 (local address 127.30.204.126:36415)
0807 00:04:56.487114 (+ 171us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:56.487119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:56.488329 (+ 1210us) server_negotiation.cc:407] Connection header received
0807 00:04:56.489795 (+ 1466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:56.489798 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:56.489858 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:56.489934 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:56.491300 (+ 1366us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.492008 (+ 708us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.493436 (+ 1428us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.493809 (+ 373us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.493927 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.496845 (+ 2918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:56.496850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:56.496854 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:56.496893 (+ 39us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:56.498543 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.499081 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.499083 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.499084 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.499161 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.499509 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.499513 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.499515 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.499768 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:56.499873 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:56.500543 (+ 670us) server_negotiation.cc:299] Negotiation successful
0807 00:04:56.500699 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:04:56.501827 23957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.487604 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:51843)
0807 00:04:56.488155 (+ 551us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:56.488236 (+ 81us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:56.489557 (+ 1321us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:56.490148 (+ 591us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:56.490156 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:56.490581 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:56.491131 (+ 550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.491148 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.492187 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.492190 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.493236 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.493252 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.494005 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.494008 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.494235 (+ 227us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.494926 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:56.494946 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:56.496614 (+ 1668us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:56.498678 (+ 2064us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:56.498685 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:56.498702 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:56.498994 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:56.499242 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:56.499245 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:56.499246 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:56.499384 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:56.499867 (+ 483us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:56.499873 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:56.500244 (+ 371us) client_negotiation.cc:769] Sending connection context
0807 00:04:56.500473 (+ 229us) client_negotiation.cc:240] Negotiation successful
0807 00:04:56.500840 (+ 367us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":123,"threads_started":1}
I20260807 00:04:56.503229 23954 heartbeater.cc:347] Connected to a master server at 127.30.204.126:36415
I20260807 00:04:56.503535 23954 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:56.504101 23954 heartbeater.cc:511] Master 127.30.204.126:36415 requested a full tablet report, sending...
I20260807 00:04:56.505347 23444 ts_manager.cc:194] Registered new tserver with Master: 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
I20260807 00:04:56.506213 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:51843
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:04:56.518296 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:36415
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:04:56.640744 23962 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:04:56.641068 23962 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:04:56.641134 23962 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:04:56.645596 23962 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:04:56.645682 23962 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:04:56.645767 23962 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:04:56.651645 23962 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23962
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:04:56.652982 23962 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:04:56.654173 23962 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:04:56.664746 23970 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:56.664863 23962 server_base.cc:1034] running on GCE node
W20260807 00:04:56.664724 23977 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:04:56.664724 23972 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:04:56.665472 23962 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:04:56.666224 23962 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:04:56.667411 23962 hybrid_clock.cc:648] HybridClock initialized: now 1786061096667389 us; error 36 us; skew 500 ppm
Aug 07 00:04:56 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061096, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:04:56.670542 23962 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:04:56.671662 23962 webserver.cc:460] Webserver started at http://127.30.204.67:35669/ using document root <none> and password file <none>
I20260807 00:04:56.672246 23962 fs_manager.cc:359] Metadata directory not provided
I20260807 00:04:56.672313 23962 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:04:56.672518 23962 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:04:56.674736 23962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "2f089e651c1242cba174dc3fbd5d2ffa"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "5e69a0469bb2ed694ecec407f3314514"
server_key_iv: "b4ce46c8e851d215c5706fecda0fb32e"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.675328 23962 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "2f089e651c1242cba174dc3fbd5d2ffa"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "5e69a0469bb2ed694ecec407f3314514"
server_key_iv: "b4ce46c8e851d215c5706fecda0fb32e"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.679431 23962 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.001s
I20260807 00:04:56.682096 23989 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.683431 23962 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.002s sys 0.000s
I20260807 00:04:56.683630 23962 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "2f089e651c1242cba174dc3fbd5d2ffa"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "5e69a0469bb2ed694ecec407f3314514"
server_key_iv: "b4ce46c8e851d215c5706fecda0fb32e"
server_key_version: "encryptionkey@0"
I20260807 00:04:56.683748 23962 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:04:56.734269 23962 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:04:56.737731 23962 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:04:56.737921 23962 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:04:56.738571 23962 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:04:56.739495 23962 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:04:56.739547 23962 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.739598 23962 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:04:56.739625 23962 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:04:56.748880 23962 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:45235
I20260807 00:04:56.748907 24175 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:45235 every 8 connection(s)
I20260807 00:04:56.749852 23962 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:04:56.757524 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 23962
I20260807 00:04:56.757684 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:04:56.757993 31537 external_mini_cluster.cc:1428] Setting key 74438a6cb198c74364e4ee2dd91b6f3e
Aug 07 00:04:56 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061096, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:04:56.765029 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.751845 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:53485 (local address 127.30.204.126:36415)
0807 00:04:56.752022 (+ 177us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:56.752027 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:56.753511 (+ 1484us) server_negotiation.cc:407] Connection header received
0807 00:04:56.754416 (+ 905us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:56.754419 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:56.754475 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:56.754552 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:56.755942 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.756505 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.757913 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.758167 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.758259 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.761272 (+ 3013us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:56.761275 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:56.761278 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:56.761313 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:56.762633 (+ 1320us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.763182 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.763184 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.763185 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.763265 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.763819 (+ 554us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.763822 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.763823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.764051 (+ 228us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:56.764145 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:56.764759 (+ 614us) server_negotiation.cc:299] Negotiation successful
0807 00:04:56.764894 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260807 00:04:56.765944 24179 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.752296 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:53485)
0807 00:04:56.753320 (+ 1024us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:56.753375 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:56.754209 (+ 834us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:56.754731 (+ 522us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:56.754738 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:56.755138 (+ 400us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:04:56.755775 (+ 637us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.755793 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.756612 (+ 819us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.756615 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.757770 (+ 1155us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.757786 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.758354 (+ 568us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.758359 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.758534 (+ 175us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.759273 (+ 739us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:04:56.759297 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:04:56.761044 (+ 1747us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:04:56.762772 (+ 1728us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:56.762779 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:56.762795 (+ 16us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:56.763101 (+ 306us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:56.763374 (+ 273us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:04:56.763377 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:04:56.763379 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:04:56.763671 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:04:56.764173 (+ 502us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:04:56.764180 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:04:56.764481 (+ 301us) client_negotiation.cc:769] Sending connection context
0807 00:04:56.764682 (+ 201us) client_negotiation.cc:240] Negotiation successful
0807 00:04:56.765002 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":797,"thread_start_us":614,"threads_started":1}
I20260807 00:04:56.767208 24176 heartbeater.cc:347] Connected to a master server at 127.30.204.126:36415
I20260807 00:04:56.767524 24176 heartbeater.cc:464] Registering TS with master...
I20260807 00:04:56.768102 24176 heartbeater.cc:511] Master 127.30.204.126:36415 requested a full tablet report, sending...
I20260807 00:04:56.769351 23444 ts_manager.cc:194] Registered new tserver with Master: 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235)
I20260807 00:04:56.770025 23444 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:53485
I20260807 00:04:56.772533 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:04:56.782226 23506 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.774634 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60366 (local address 127.30.204.126:36415)
0807 00:04:56.774772 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:56.774775 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:56.774931 (+ 156us) server_negotiation.cc:407] Connection header received
0807 00:04:56.775088 (+ 157us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:56.775091 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:56.775142 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:56.775215 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:04:56.776007 (+ 792us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.776544 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.777414 (+ 870us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.777646 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.777749 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.779075 (+ 1326us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:04:56.779078 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:04:56.779080 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:04:56.779123 (+ 43us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:04:56.780489 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.780946 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.780949 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.780950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.781052 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:04:56.781326 (+ 274us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:04:56.781329 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:04:56.781330 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:04:56.781620 (+ 290us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:04:56.781735 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:56.781967 (+ 232us) server_negotiation.cc:299] Negotiation successful
0807 00:04:56.782092 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:04:56.785298 23444 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60366:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:04:56.788625 23444 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:04:56.804263 24195 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.797632 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:45235 (local address 127.0.0.1:36598)
0807 00:04:56.798152 (+ 520us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:56.798205 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:56.798340 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:56.799525 (+ 1185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:56.799531 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:56.799556 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:56.799706 (+ 150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.799714 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.801314 (+ 1600us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.801317 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.802699 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.802709 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.803846 (+ 1137us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.803850 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.803931 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.803956 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:04:56.804002 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 00:04:56.804103 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":186,"mutex_wait_us":331,"thread_start_us":84,"threads_started":1}
I20260807 00:04:56.804273 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.798458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58158 (local address 127.30.204.66:44529)
0807 00:04:56.798911 (+ 453us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:56.798925 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:56.799726 (+ 801us) server_negotiation.cc:407] Connection header received
0807 00:04:56.799807 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:56.799812 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:56.799962 (+ 150us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:56.800092 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:56.800494 (+ 402us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.801267 (+ 773us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.802883 (+ 1616us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.803501 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.803608 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.803784 (+ 176us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:56.803989 (+ 205us) server_negotiation.cc:299] Negotiation successful
0807 00:04:56.804102 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":110,"threads_started":1}
I20260807 00:04:56.804548 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.798352 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36598 (local address 127.30.204.67:45235)
0807 00:04:56.799073 (+ 721us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:56.799081 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:56.799101 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:04:56.799167 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:56.799171 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:56.799339 (+ 168us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:56.799527 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:56.799825 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.801155 (+ 1330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.802826 (+ 1671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.803695 (+ 869us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.804097 (+ 402us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.804244 (+ 147us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:56.804293 (+ 49us) server_negotiation.cc:299] Negotiation successful
0807 00:04:56.804389 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":644,"thread_start_us":138,"threads_started":1}
I20260807 00:04:56.805176 24196 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.797954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:44529 (local address 127.0.0.1:58158)
0807 00:04:56.799598 (+ 1644us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:56.799610 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:56.799687 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:56.800166 (+ 479us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:56.800169 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:56.800194 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:56.800378 (+ 184us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.800388 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.801467 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.801470 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.802760 (+ 1290us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.802770 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.803619 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.803623 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.803788 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.803835 (+ 47us) client_negotiation.cc:769] Sending connection context
0807 00:04:56.803945 (+ 110us) client_negotiation.cc:240] Negotiation successful
0807 00:04:56.805055 (+ 1110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1587,"mutex_wait_us":318,"thread_start_us":97,"threads_started":1}
I20260807 00:04:56.805348 24197 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.797743 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41603 (local address 127.0.0.1:57936)
0807 00:04:56.799288 (+ 1545us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:56.799300 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:56.799391 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:56.800012 (+ 621us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:56.800015 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:56.800033 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:56.800126 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.800133 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.801489 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.801492 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.803730 (+ 2238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:56.803740 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.804951 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.804954 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:56.805042 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.805064 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:04:56.805143 (+ 79us) client_negotiation.cc:240] Negotiation successful
0807 00:04:56.805215 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1459,"mutex_wait_us":103,"thread_start_us":128,"threads_started":1}
I20260807 00:04:56.806263 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:56.798766 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.30.204.65:41603)
0807 00:04:56.799031 (+ 265us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:56.799038 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:56.799389 (+ 351us) server_negotiation.cc:407] Connection header received
0807 00:04:56.799515 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:56.799521 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:56.799730 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:56.799866 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:56.800260 (+ 394us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.801373 (+ 1113us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.803861 (+ 2488us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:56.804656 (+ 795us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:56.804760 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:56.804894 (+ 134us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:56.806000 (+ 1106us) server_negotiation.cc:299] Negotiation successful
0807 00:04:56.806113 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":175,"thread_start_us":86,"threads_started":1}
I20260807 00:04:56.808353 23849 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd437d1a4c548778a8f934c15c13ef6 (DEFAULT_TABLE table=test-table [id=7a97c127c1cc40db96e321c9377f3eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:56.808353 24068 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd437d1a4c548778a8f934c15c13ef6 (DEFAULT_TABLE table=test-table [id=7a97c127c1cc40db96e321c9377f3eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:56.808636 23643 tablet_service.cc:1467] Processing CreateTablet for tablet 0bd437d1a4c548778a8f934c15c13ef6 (DEFAULT_TABLE table=test-table [id=7a97c127c1cc40db96e321c9377f3eca]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:04:56.809533 23643 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd437d1a4c548778a8f934c15c13ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:56.809551 23849 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd437d1a4c548778a8f934c15c13ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:56.809559 24068 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 0bd437d1a4c548778a8f934c15c13ef6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:04:56.815974 24202 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Bootstrap starting.
I20260807 00:04:56.817643 24205 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Bootstrap starting.
I20260807 00:04:56.817643 24203 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Bootstrap starting.
I20260807 00:04:56.818975 24202 tablet_bootstrap.cc:654] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:56.819773 24202 log.cc:824] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:56.820334 24205 tablet_bootstrap.cc:654] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:56.820360 24203 tablet_bootstrap.cc:654] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Neither blocks nor log segments found. Creating new log.
I20260807 00:04:56.821228 24203 log.cc:824] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:56.821218 24205 log.cc:824] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Log is configured to *not* fsync() on all Append() calls
I20260807 00:04:56.821818 24202 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: No bootstrap required, opened a new log
I20260807 00:04:56.822050 24202 ts_tablet_manager.cc:1397] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:04:56.823357 24205 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: No bootstrap required, opened a new log
I20260807 00:04:56.823633 24205 ts_tablet_manager.cc:1397] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Time spent bootstrapping tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:04:56.823769 24203 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: No bootstrap required, opened a new log
I20260807 00:04:56.824091 24203 ts_tablet_manager.cc:1397] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Time spent bootstrapping tablet: real 0.007s user 0.006s sys 0.000s
I20260807 00:04:56.826296 24202 raft_consensus.cc:348] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:56.826546 24202 raft_consensus.cc:374] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:56.826627 24202 raft_consensus.cc:729] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743a6966a0454730a1bce6148ab272ac, State: Initialized, Role: FOLLOWER
I20260807 00:04:56.827092 24202 consensus_queue.cc:260] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:56.827821 23746 heartbeater.cc:503] Master 127.30.204.126:36415 was elected leader, sending a full tablet report...
I20260807 00:04:56.827735 24203 raft_consensus.cc:348] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:56.827689 24205 raft_consensus.cc:348] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:56.827975 24205 raft_consensus.cc:374] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:56.827975 24203 raft_consensus.cc:374] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:04:56.828022 24203 raft_consensus.cc:729] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f089e651c1242cba174dc3fbd5d2ffa, State: Initialized, Role: FOLLOWER
I20260807 00:04:56.828022 24205 raft_consensus.cc:729] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053baa0d2ba14e6baef7f84a8eb38106, State: Initialized, Role: FOLLOWER
I20260807 00:04:56.828213 24202 ts_tablet_manager.cc:1428] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:04:56.828500 24203 consensus_queue.cc:260] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:56.828500 24205 consensus_queue.cc:260] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:56.829347 23954 heartbeater.cc:503] Master 127.30.204.126:36415 was elected leader, sending a full tablet report...
I20260807 00:04:56.829402 24176 heartbeater.cc:503] Master 127.30.204.126:36415 was elected leader, sending a full tablet report...
I20260807 00:04:56.829531 24205 ts_tablet_manager.cc:1428] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Time spent starting tablet: real 0.006s user 0.005s sys 0.000s
I20260807 00:04:56.829531 24203 ts_tablet_manager.cc:1428] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Time spent starting tablet: real 0.005s user 0.005s sys 0.000s
W20260807 00:04:56.977640 23747 tablet.cc:2369] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:04:56.986030 23956 tablet.cc:2369] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:04:57.000957 24177 tablet.cc:2369] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:04:57.147302 24211 raft_consensus.cc:482] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:04:57.147585 24211 raft_consensus.cc:504] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:57.148792 24211 leader_election.cc:288] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235), 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
I20260807 00:04:57.152931 24200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:57.149174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:40541 (local address 127.30.204.66:44529)
0807 00:04:57.149318 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:57.149323 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:57.149598 (+ 275us) server_negotiation.cc:407] Connection header received
0807 00:04:57.149801 (+ 203us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:57.149805 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:57.149855 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:57.149924 (+ 69us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:57.150226 (+ 302us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.150745 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.151926 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.152326 (+ 400us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.152441 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:57.152500 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:57.152710 (+ 210us) server_negotiation.cc:299] Negotiation successful
0807 00:04:57.152794 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:04:57.152931 24198 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:57.149294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:36733 (local address 127.30.204.67:45235)
0807 00:04:57.149414 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:57.149418 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:57.149694 (+ 276us) server_negotiation.cc:407] Connection header received
0807 00:04:57.149807 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:57.149809 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:57.149850 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:57.149914 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:04:57.150215 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.150739 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.151925 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.152331 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.152443 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:57.152500 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:57.152713 (+ 213us) server_negotiation.cc:299] Negotiation successful
0807 00:04:57.152788 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 00:04:57.152974 24215 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:57.149090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:45235 (local address 127.30.204.65:36733)
0807 00:04:57.149607 (+ 517us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:57.149619 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:57.149718 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:57.149942 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:57.149944 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:57.149969 (+ 25us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:57.150108 (+ 139us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:57.150114 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.150851 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.150855 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:57.151804 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:57.151814 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.152448 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.152452 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:57.152580 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:57.152619 (+ 39us) client_negotiation.cc:769] Sending connection context
0807 00:04:57.152684 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 00:04:57.152782 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":440,"mutex_wait_us":64,"spinlock_wait_cycles":14464,"thread_start_us":86,"threads_started":1}
I20260807 00:04:57.152973 24216 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:57.149090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:44529 (local address 127.30.204.65:40541)
0807 00:04:57.149493 (+ 403us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:57.149506 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:04:57.149654 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:04:57.149943 (+ 289us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:04:57.149946 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:04:57.149977 (+ 31us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:04:57.150111 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:57.150124 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.150848 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.150852 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:57.151801 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:04:57.151812 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.152444 (+ 632us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.152448 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:04:57.152546 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:57.152612 (+ 66us) client_negotiation.cc:769] Sending connection context
0807 00:04:57.152681 (+ 69us) client_negotiation.cc:240] Negotiation successful
0807 00:04:57.152787 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":336,"mutex_wait_us":70,"spinlock_wait_cycles":5376,"thread_start_us":122,"threads_started":1}
I20260807 00:04:57.153739 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd437d1a4c548778a8f934c15c13ef6" candidate_uuid: "743a6966a0454730a1bce6148ab272ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" is_pre_election: true
I20260807 00:04:57.153851 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd437d1a4c548778a8f934c15c13ef6" candidate_uuid: "743a6966a0454730a1bce6148ab272ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053baa0d2ba14e6baef7f84a8eb38106" is_pre_election: true
I20260807 00:04:57.154071 24095 raft_consensus.cc:2454] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 743a6966a0454730a1bce6148ab272ac in term 0.
I20260807 00:04:57.154167 23880 raft_consensus.cc:2454] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 743a6966a0454730a1bce6148ab272ac in term 0.
I20260807 00:04:57.154608 23576 leader_election.cc:302] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 053baa0d2ba14e6baef7f84a8eb38106, 2f089e651c1242cba174dc3fbd5d2ffa, 743a6966a0454730a1bce6148ab272ac; no voters:
I20260807 00:04:57.154875 24211 raft_consensus.cc:2781] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:04:57.154963 24211 raft_consensus.cc:482] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:04:57.154996 24211 raft_consensus.cc:3037] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:57.156236 24211 raft_consensus.cc:504] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:57.156628 24211 leader_election.cc:288] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 1 election: Requested vote from peers 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235), 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
I20260807 00:04:57.157094 23880 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd437d1a4c548778a8f934c15c13ef6" candidate_uuid: "743a6966a0454730a1bce6148ab272ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053baa0d2ba14e6baef7f84a8eb38106"
I20260807 00:04:57.157232 23880 raft_consensus.cc:3037] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:57.158344 23880 raft_consensus.cc:2454] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 743a6966a0454730a1bce6148ab272ac in term 1.
I20260807 00:04:57.158660 24095 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "0bd437d1a4c548778a8f934c15c13ef6" candidate_uuid: "743a6966a0454730a1bce6148ab272ac" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2f089e651c1242cba174dc3fbd5d2ffa"
I20260807 00:04:57.158769 23574 leader_election.cc:302] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 053baa0d2ba14e6baef7f84a8eb38106, 743a6966a0454730a1bce6148ab272ac; no voters:
I20260807 00:04:57.158826 24095 raft_consensus.cc:3037] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:04:57.158989 24211 raft_consensus.cc:2781] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:04:57.159309 24211 raft_consensus.cc:686] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 1 LEADER]: Becoming Leader. State: Replica: 743a6966a0454730a1bce6148ab272ac, State: Running, Role: LEADER
I20260807 00:04:57.159741 24211 consensus_queue.cc:237] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:04:57.160300 24095 raft_consensus.cc:2454] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 743a6966a0454730a1bce6148ab272ac in term 1.
I20260807 00:04:57.163506 23444 catalog_manager.cc:5495] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac reported cstate change: term changed from 0 to 1, leader changed from <none> to 743a6966a0454730a1bce6148ab272ac (127.30.204.65). New cstate: current_term: 1 leader_uuid: "743a6966a0454730a1bce6148ab272ac" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } health_report { overall_health: UNKNOWN } } }
I20260807 00:04:57.261853 24201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:57.258183 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57952 (local address 127.30.204.65:41603)
0807 00:04:57.258357 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:04:57.258361 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:04:57.258375 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:04:57.258461 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:04:57.258463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:04:57.258520 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:04:57.258613 (+ 93us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:04:57.258950 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.259488 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.260418 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:04:57.260635 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:04:57.260712 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:04:57.260983 (+ 271us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:04:57.261442 (+ 459us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:04:57.261558 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0807 00:04:57.261627 (+ 69us) server_negotiation.cc:299] Negotiation successful
0807 00:04:57.261697 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260807 00:04:57.268473 23880 raft_consensus.cc:1261] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Refusing update from remote peer 743a6966a0454730a1bce6148ab272ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:57.268472 24095 raft_consensus.cc:1261] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 FOLLOWER]: Refusing update from remote peer 743a6966a0454730a1bce6148ab272ac: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:04:57.269289 24211 consensus_queue.cc:1035] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:57.269526 24217 consensus_queue.cc:1035] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [LEADER]: Connected to new peer: Peer: permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:04:57.278376 24222 mvcc.cc:205] Tried to move back new op lower bound from 7315706254402748416 to 7315706253969944576. Current Snapshot: MvccSnapshot[applied={T|T < 7315706254402748416}]
I20260807 00:04:57.286386 23441 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:57.286597 23441 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:60366:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:04:57.289433 23441 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Sending DeleteTablet for 3 replicas of tablet 0bd437d1a4c548778a8f934c15c13ef6
I20260807 00:04:57.290405 24068 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bd437d1a4c548778a8f934c15c13ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:57 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:36598
I20260807 00:04:57.290405 23643 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bd437d1a4c548778a8f934c15c13ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:57 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:57936
I20260807 00:04:57.290511 23849 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bd437d1a4c548778a8f934c15c13ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:57 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58158
I20260807 00:04:57.291134 24236 tablet_replica.cc:330] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: stopping tablet replica
I20260807 00:04:57.291173 24237 tablet_replica.cc:330] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: stopping tablet replica
I20260807 00:04:57.291217 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 23548
I20260807 00:04:57.305784 24236 raft_consensus.cc:2229] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:04:57.306267 24236 raft_consensus.cc:2258] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:57.309127 24236 ts_tablet_manager.cc:1905] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:04:57.311771 24238 tablet_replica.cc:330] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: stopping tablet replica
I20260807 00:04:57.312131 24238 raft_consensus.cc:2229] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:04:57.312297 24236 ts_tablet_manager.cc:1918] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:04:57.312428 24236 log.cc:1196] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/0bd437d1a4c548778a8f934c15c13ef6
I20260807 00:04:57.312522 24238 raft_consensus.cc:2258] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:04:57.312747 24236 ts_tablet_manager.cc:1939] T 0bd437d1a4c548778a8f934c15c13ef6 P 2f089e651c1242cba174dc3fbd5d2ffa: Deleting consensus metadata
I20260807 00:04:57.313951 24238 ts_tablet_manager.cc:1905] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:04:57.314261 23420 catalog_manager.cc:4841] TS 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235): tablet 0bd437d1a4c548778a8f934c15c13ef6 (table test-table [id=7a97c127c1cc40db96e321c9377f3eca]) successfully deleted
I20260807 00:04:57.315222 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 23753
W20260807 00:04:57.315398 23415 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:41603 (error 108)
I20260807 00:04:57.323493 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:57.322830 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41603 (local address 127.0.0.1:57956)
0807 00:04:57.323270 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:57.323353 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":361,"thread_start_us":66,"threads_started":1}
W20260807 00:04:57.323740 23415 catalog_manager.cc:4584] TS 743a6966a0454730a1bce6148ab272ac (127.30.204.65:41603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 0bd437d1a4c548778a8f934c15c13ef6: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41603: connect: Connection refused (error 111)
I20260807 00:04:57.328704 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 23962
I20260807 00:04:57.342901 24243 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:04:57.329022 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:44529 (local address 127.0.0.1:58162)
0807 00:04:57.342682 (+ 13660us) negotiation.cc:107] Waiting for socket to connect
0807 00:04:57.342790 (+ 108us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.66:44529: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":13489}
I20260807 00:04:57.348371 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 23386
I20260807 00:05:17.358474 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36415
--webserver_interface=127.30.204.126
--webserver_port=43239
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:36415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:17.477327 24244 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:17.477641 24244 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:17.477702 24244 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:17.481876 24244 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 00:05:17.481956 24244 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 00:05:17.481977 24244 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 00:05:17.481997 24244 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:05:17.482020 24244 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:17.482045 24244 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:17.482075 24244 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:05:17.482098 24244 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:05:17.487811 24244 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:36415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24244
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:17.489017 24244 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:17.490144 24244 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:17.504936 24252 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:17.504915 24255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:17.504907 24251 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:17.505254 24244 server_base.cc:1034] running on GCE node
I20260807 00:05:17.505893 24244 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:17.507187 24244 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:17.508369 24244 hybrid_clock.cc:648] HybridClock initialized: now 1786061117508320 us; error 56 us; skew 500 ppm
Aug 07 00:05:17 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061117, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:17.514176 24244 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:05:17.515270 24244 webserver.cc:460] Webserver started at http://127.30.204.126:43239/ using document root <none> and password file <none>
I20260807 00:05:17.515923 24244 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:17.516016 24244 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:17.520393 24244 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:05:17.523031 24271 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:17.524526 24244 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:17.524706 24244 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "a19425ff05a345668eb4b1729cffe2a0"
format_stamp: "Formatted at 2026-08-07 00:04:55 on dist-test-slave-pww0"
server_key: "6672a34c216625ef5484a239633aeaaf"
server_key_iv: "d44d9a3fdedca6c68b5390c2b13332b9"
server_key_version: "encryptionkey@0"
I20260807 00:05:17.525383 24244 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:17.567517 24244 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:17.571115 24244 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:17.571352 24244 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:17.580183 24244 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:36415
I20260807 00:05:17.580480 24361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:36415 every 8 connection(s)
I20260807 00:05:17.581477 24244 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:05:17.586548 24362 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:05:17.587788 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 24244
I20260807 00:05:17.588330 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:41603
--local_ip_for_outbound_sockets=127.30.204.65
--tserver_master_addrs=127.30.204.126:36415
--webserver_port=43971
--webserver_interface=127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:05:17.588977 24362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Bootstrap starting.
I20260807 00:05:17.608180 24362 log.cc:824] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:17.618432 24362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:05:17.619035 24362 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Bootstrap complete.
I20260807 00:05:17.623729 24362 raft_consensus.cc:348] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:17.624281 24362 raft_consensus.cc:729] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19425ff05a345668eb4b1729cffe2a0, State: Initialized, Role: FOLLOWER
I20260807 00:05:17.624840 24362 consensus_queue.cc:260] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:17.624970 24362 raft_consensus.cc:388] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:05:17.625046 24362 raft_consensus.cc:482] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:05:17.625154 24362 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 1 FOLLOWER]: Advancing to term 2
I20260807 00:05:17.627334 24362 raft_consensus.cc:504] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:17.627740 24362 leader_election.cc:302] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19425ff05a345668eb4b1729cffe2a0; no voters:
I20260807 00:05:17.628100 24362 leader_election.cc:288] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260807 00:05:17.628214 24367 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 FOLLOWER]: Leader election won for term 2
I20260807 00:05:17.628474 24367 raft_consensus.cc:686] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 LEADER]: Becoming Leader. State: Replica: a19425ff05a345668eb4b1729cffe2a0, State: Running, Role: LEADER
I20260807 00:05:17.628877 24367 consensus_queue.cc:237] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:17.629603 24362 sys_catalog.cc:564] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:05:17.630046 24367 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a19425ff05a345668eb4b1729cffe2a0. Latest consensus state: current_term: 2 leader_uuid: "a19425ff05a345668eb4b1729cffe2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } } }
I20260807 00:05:17.630179 24367 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:17.630448 24369 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "a19425ff05a345668eb4b1729cffe2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } } }
I20260807 00:05:17.630656 24369 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:17.630980 24372 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:05:17.636329 24372 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:05:17.637189 24372 catalog_manager.cc:1259] Loaded cluster ID: fe43b79711d04b5c84ee192046b002e1
I20260807 00:05:17.637254 24372 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:05:17.639158 24372 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:05:17.639971 24372 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Loaded TSK: 0
I20260807 00:05:17.650736 24372 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Generated new TSK 1
I20260807 00:05:17.651037 24372 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:05:17.742540 24365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:17.742945 24365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:17.743009 24365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:17.749161 24365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:17.749271 24365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:17.749365 24365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:05:17.757177 24365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:41603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=43971
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:17.758627 24365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:17.760097 24365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:17.772250 24407 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:17.772248 24414 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:17.772274 24406 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:17.772977 24365 server_base.cc:1034] running on GCE node
I20260807 00:05:17.773510 24365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:17.774447 24365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:17.775796 24365 hybrid_clock.cc:648] HybridClock initialized: now 1786061117775757 us; error 54 us; skew 500 ppm
Aug 07 00:05:17 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061117, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:17.779779 24365 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:05:17.781164 24365 webserver.cc:460] Webserver started at http://127.30.204.65:43971/ using document root <none> and password file <none>
I20260807 00:05:17.781985 24365 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:17.782122 24365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:17.788239 24365 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:05:17.791311 24422 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:17.793088 24365 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:05:17.793247 24365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "743a6966a0454730a1bce6148ab272ac"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "b6fea95a4cfdb65006cf0464370210dc"
server_key_iv: "53d4810c9cecb98383a0cedd1f38f50e"
server_key_version: "encryptionkey@0"
I20260807 00:05:17.794061 24365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:17.829416 24365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:17.833293 24365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:17.833519 24365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:17.834193 24365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:17.835589 24431 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:05:17.840090 24365 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:05:17.840188 24365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.001s sys 0.000s
I20260807 00:05:17.840274 24365 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:05:17.842271 24365 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:05:17.842335 24365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.000s sys 0.000s
I20260807 00:05:17.842753 24431 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Bootstrap starting.
I20260807 00:05:17.852507 24365 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:41603
I20260807 00:05:17.852583 24595 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:41603 every 8 connection(s)
I20260807 00:05:17.853606 24365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:05:17.858327 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 24365
I20260807 00:05:17.858889 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:44529
--local_ip_for_outbound_sockets=127.30.204.66
--tserver_master_addrs=127.30.204.126:36415
--webserver_port=34273
--webserver_interface=127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:05:17.864486 24431 log.cc:824] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Log is configured to *not* fsync() on all Append() calls
Aug 07 00:05:17 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061117, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:17.873440 24600 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:17.856106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:55967)
0807 00:05:17.856732 (+ 626us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:17.856778 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:17.857645 (+ 867us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:17.858901 (+ 1256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:17.858909 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:17.859613 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:17.860243 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:17.860262 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.861841 (+ 1579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.861845 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:17.862762 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:17.862774 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.863594 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.863599 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:17.863813 (+ 214us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:17.864541 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:17.864561 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:17.866556 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:17.869363 (+ 2807us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:17.869371 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:17.869390 (+ 19us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:17.869779 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:17.870228 (+ 449us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:17.870231 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:17.870233 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:17.870449 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:17.871232 (+ 783us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:17.871239 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:17.871626 (+ 387us) client_negotiation.cc:769] Sending connection context
0807 00:05:17.871861 (+ 235us) client_negotiation.cc:240] Negotiation successful
0807 00:05:17.872282 (+ 421us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":416,"thread_start_us":211,"threads_started":1}
I20260807 00:05:17.873504 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:17.856082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:55967 (local address 127.30.204.126:36415)
0807 00:05:17.856471 (+ 389us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:17.856492 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:17.856913 (+ 421us) server_negotiation.cc:407] Connection header received
0807 00:05:17.858094 (+ 1181us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:17.858106 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:17.858445 (+ 339us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:17.858816 (+ 371us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:17.860454 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.861654 (+ 1200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.862901 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.863421 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.863609 (+ 188us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:17.866830 (+ 3221us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:17.866838 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:17.866856 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:17.866896 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:17.869204 (+ 2308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:17.869930 (+ 726us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:17.869937 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:17.869943 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:17.870112 (+ 169us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:17.870559 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:17.870563 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:17.870565 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:17.871087 (+ 522us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:17.871231 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:17.872008 (+ 777us) server_negotiation.cc:299] Negotiation successful
0807 00:05:17.872337 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":246,"thread_start_us":124,"threads_started":1}
I20260807 00:05:17.874099 24431 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:05:17.874651 24431 tablet_bootstrap.cc:492] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Bootstrap complete.
I20260807 00:05:17.876060 24431 ts_tablet_manager.cc:1397] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Time spent bootstrapping tablet: real 0.034s user 0.019s sys 0.012s
I20260807 00:05:17.876919 24596 heartbeater.cc:347] Connected to a master server at 127.30.204.126:36415
I20260807 00:05:17.877189 24596 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:17.877831 24596 heartbeater.cc:511] Master 127.30.204.126:36415 requested a full tablet report, sending...
I20260807 00:05:17.879184 24431 raft_consensus.cc:348] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:05:17.879721 24431 raft_consensus.cc:729] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743a6966a0454730a1bce6148ab272ac, State: Initialized, Role: FOLLOWER
I20260807 00:05:17.880270 24431 consensus_queue.cc:260] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } }
I20260807 00:05:17.881016 24297 ts_manager.cc:194] Registered new tserver with Master: 743a6966a0454730a1bce6148ab272ac (127.30.204.65:41603)
I20260807 00:05:17.881450 24431 ts_tablet_manager.cc:1428] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Time spent starting tablet: real 0.005s user 0.004s sys 0.001s
I20260807 00:05:17.884819 24297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:55967
I20260807 00:05:17.887658 24596 heartbeater.cc:503] Master 127.30.204.126:36415 was elected leader, sending a full tablet report...
Aug 07 00:05:17 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061117, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.65@KRBTEST.COM
I20260807 00:05:17.896006 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:17.884354 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41603 (local address 127.0.0.1:35842)
0807 00:05:17.885190 (+ 836us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:17.885238 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:17.885405 (+ 167us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:17.885949 (+ 544us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:17.885955 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:17.886439 (+ 484us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:17.886580 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:17.886589 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.888013 (+ 1424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.888017 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:17.888786 (+ 769us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:17.888797 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.889405 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.889409 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:17.889524 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:17.890157 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:17.890170 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:17.891921 (+ 1751us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:17.894207 (+ 2286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:17.894217 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:17.894221 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:17.894484 (+ 263us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:17.894792 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:17.894794 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:17.894795 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:17.894876 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:17.895380 (+ 504us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:17.895385 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:17.895557 (+ 172us) client_negotiation.cc:769] Sending connection context
0807 00:05:17.895684 (+ 127us) client_negotiation.cc:240] Negotiation successful
0807 00:05:17.895863 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":654,"thread_start_us":125,"threads_started":1}
I20260807 00:05:17.896147 24605 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:17.884466 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35842 (local address 127.30.204.65:41603)
0807 00:05:17.884908 (+ 442us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:17.884921 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:17.885368 (+ 447us) server_negotiation.cc:407] Connection header received
0807 00:05:17.885560 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:17.885565 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:17.885788 (+ 223us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:17.885934 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:17.886732 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.887867 (+ 1135us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.888902 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:17.889284 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:17.889695 (+ 411us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:17.892088 (+ 2393us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:17.892107 (+ 19us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:17.892120 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:17.892151 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:17.894061 (+ 1910us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:17.894598 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:17.894604 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:17.894606 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:17.894703 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:17.894947 (+ 244us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:17.894949 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:17.894951 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:17.895280 (+ 329us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:17.895409 (+ 129us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:17.895784 (+ 375us) server_negotiation.cc:299] Negotiation successful
0807 00:05:17.895971 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":337,"thread_start_us":93,"threads_started":1}
I20260807 00:05:17.897185 24498 tablet_service.cc:1514] Processing DeleteTablet for tablet 0bd437d1a4c548778a8f934c15c13ef6 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:04:57 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:35842
I20260807 00:05:17.898097 24606 tablet_replica.cc:330] stopping tablet replica
I20260807 00:05:17.898423 24606 raft_consensus.cc:2229] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:05:17.898675 24606 raft_consensus.cc:2258] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:17.899824 24606 ts_tablet_manager.cc:1905] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:05:17.903838 24606 ts_tablet_manager.cc:1918] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:05:17.903954 24606 log.cc:1196] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal/wals/0bd437d1a4c548778a8f934c15c13ef6
I20260807 00:05:17.904207 24606 ts_tablet_manager.cc:1939] T 0bd437d1a4c548778a8f934c15c13ef6 P 743a6966a0454730a1bce6148ab272ac: Deleting consensus metadata
I20260807 00:05:17.906315 24272 catalog_manager.cc:4841] TS 743a6966a0454730a1bce6148ab272ac (127.30.204.65:41603): tablet 0bd437d1a4c548778a8f934c15c13ef6 (table test-table [id=7a97c127c1cc40db96e321c9377f3eca]) successfully deleted
W20260807 00:05:17.988960 24601 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:17.989287 24601 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:17.989354 24601 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:17.993655 24601 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:17.993757 24601 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:17.993844 24601 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:05:17.999609 24601 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:44529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=34273
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24601
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:18.000909 24601 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:18.002027 24601 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:18.012359 24618 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:18.012360 24617 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:18.012482 24620 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:18.012887 24601 server_base.cc:1034] running on GCE node
I20260807 00:05:18.013321 24601 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:18.014005 24601 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:18.015198 24601 hybrid_clock.cc:648] HybridClock initialized: now 1786061118015162 us; error 51 us; skew 500 ppm
Aug 07 00:05:18 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061118, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:18.018282 24601 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:05:18.019330 24601 webserver.cc:460] Webserver started at http://127.30.204.66:34273/ using document root <none> and password file <none>
I20260807 00:05:18.019953 24601 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:18.020035 24601 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:18.024350 24601 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:05:18.026878 24629 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:18.028363 24601 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:18.028506 24601 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "053baa0d2ba14e6baef7f84a8eb38106"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "25deb440e738f0fbc1b294dcf09ff545"
server_key_iv: "86907273a3f2b459bdc3fb87147f9ba7"
server_key_version: "encryptionkey@0"
I20260807 00:05:18.029156 24601 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:18.060395 24601 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:18.063891 24601 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:18.064100 24601 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:18.064781 24601 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:18.066031 24640 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260807 00:05:18.070325 24640 ts_tablet_manager.cc:1852] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260807 00:05:18.070768 24640 ts_tablet_manager.cc:1905] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:05:18.075512 24640 ts_tablet_manager.cc:1918] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:05:18.075672 24640 log.cc:1196] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/0bd437d1a4c548778a8f934c15c13ef6
I20260807 00:05:18.076037 24640 ts_tablet_manager.cc:1939] T 0bd437d1a4c548778a8f934c15c13ef6 P 053baa0d2ba14e6baef7f84a8eb38106: Deleting consensus metadata
I20260807 00:05:18.076407 24601 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260807 00:05:18.076498 24601 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.011s user 0.001s sys 0.000s
I20260807 00:05:18.076579 24601 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:18.076608 24601 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:18.086561 24601 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:44529
I20260807 00:05:18.086644 24818 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:44529 every 8 connection(s)
I20260807 00:05:18.087621 24601 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:05:18.090296 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 24601
I20260807 00:05:18.090758 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:45235
--local_ip_for_outbound_sockets=127.30.204.67
--tserver_master_addrs=127.30.204.126:36415
--webserver_port=35669
--webserver_interface=127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:05:18 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061118, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:18.103760 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.089680 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:33171 (local address 127.30.204.126:36415)
0807 00:05:18.089866 (+ 186us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.089871 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.090828 (+ 957us) server_negotiation.cc:407] Connection header received
0807 00:05:18.091875 (+ 1047us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.091879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.091942 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.092040 (+ 98us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:18.093636 (+ 1596us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.094168 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.095385 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.095697 (+ 312us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.095809 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.099142 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:18.099145 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:18.099147 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:18.099181 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:18.100878 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.101637 (+ 759us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.101640 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.101641 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.101737 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.102220 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.102223 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.102224 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.102487 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:18.102573 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.103388 (+ 815us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.103579 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260807 00:05:18.104832 24822 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.090086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:33171)
0807 00:05:18.090635 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:18.090702 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:18.091652 (+ 950us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:18.092206 (+ 554us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:18.092213 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:18.092664 (+ 451us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:18.093425 (+ 761us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.093448 (+ 23us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.094317 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.094320 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.095243 (+ 923us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.095254 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.095913 (+ 659us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.095917 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.096162 (+ 245us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.096871 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:18.096897 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:18.098870 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:18.101021 (+ 2151us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:18.101030 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:18.101053 (+ 23us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:18.101490 (+ 437us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:18.101875 (+ 385us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:18.101878 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:18.101880 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:18.102105 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:18.102610 (+ 505us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:18.102618 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:18.103027 (+ 409us) client_negotiation.cc:769] Sending connection context
0807 00:05:18.103284 (+ 257us) client_negotiation.cc:240] Negotiation successful
0807 00:05:18.103717 (+ 433us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":141,"threads_started":1}
I20260807 00:05:18.106149 24820 heartbeater.cc:347] Connected to a master server at 127.30.204.126:36415
I20260807 00:05:18.106441 24820 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:18.107069 24820 heartbeater.cc:511] Master 127.30.204.126:36415 requested a full tablet report, sending...
I20260807 00:05:18.108415 24297 ts_manager.cc:194] Registered new tserver with Master: 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
I20260807 00:05:18.109050 24297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:33171
W20260807 00:05:18.214747 24825 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:18.215085 24825 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:18.215152 24825 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:18.219398 24825 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:18.219506 24825 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:18.219591 24825 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:05:18.225293 24825 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:45235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=35669
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24825
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:18.226507 24825 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:18.227685 24825 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:18.238512 24833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:18.238621 24825 server_base.cc:1034] running on GCE node
W20260807 00:05:18.238507 24835 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:18.238905 24840 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:18.239414 24825 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:18.240254 24825 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:18.241431 24825 hybrid_clock.cc:648] HybridClock initialized: now 1786061118241407 us; error 35 us; skew 500 ppm
Aug 07 00:05:18 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061118, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:18.244508 24825 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:05:18.245545 24825 webserver.cc:460] Webserver started at http://127.30.204.67:35669/ using document root <none> and password file <none>
I20260807 00:05:18.246126 24825 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:18.246206 24825 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:18.250411 24825 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:05:18.252875 24853 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:18.254428 24825 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:05:18.254561 24825 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "2f089e651c1242cba174dc3fbd5d2ffa"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "5e69a0469bb2ed694ecec407f3314514"
server_key_iv: "b4ce46c8e851d215c5706fecda0fb32e"
server_key_version: "encryptionkey@0"
I20260807 00:05:18.255208 24825 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:18.271675 24825 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:18.275184 24825 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:18.275379 24825 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:18.276109 24825 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:18.277079 24825 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:18.277133 24825 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:18.277177 24825 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:18.277206 24825 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:18.286119 24825 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:45235
I20260807 00:05:18.286154 25027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:45235 every 8 connection(s)
I20260807 00:05:18.287199 24825 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:05:18.289562 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 24825
Aug 07 00:05:18 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061118, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:18.298573 25035 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.290278 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52524 (local address 127.30.204.126:36415)
0807 00:05:18.290539 (+ 261us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.290544 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.290914 (+ 370us) server_negotiation.cc:407] Connection header received
0807 00:05:18.291031 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.291034 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.291086 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.291193 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:18.292035 (+ 842us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.292837 (+ 802us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.293677 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.293909 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.293999 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.295417 (+ 1418us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:18.295420 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:18.295422 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:18.295488 (+ 66us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:18.296882 (+ 1394us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.297319 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.297323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.297324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.297425 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.297697 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.297699 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.297700 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.297882 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:18.298030 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.298253 (+ 223us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.298395 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":93,"threads_started":1}
I20260807 00:05:18.302045 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.289220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:55185 (local address 127.30.204.126:36415)
0807 00:05:18.289394 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.289399 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.290626 (+ 1227us) server_negotiation.cc:407] Connection header received
0807 00:05:18.291726 (+ 1100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.291729 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.291778 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.291849 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:18.293314 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.293785 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.295086 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.295309 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.295394 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.298115 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:18.298118 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:18.298120 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:18.298141 (+ 21us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:18.299744 (+ 1603us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.300411 (+ 667us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.300414 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.300416 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.300504 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.300882 (+ 378us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.300885 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.300887 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.301069 (+ 182us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:18.301155 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.301748 (+ 593us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.301880 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 00:05:18.302949 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.289638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:55185)
0807 00:05:18.290461 (+ 823us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:18.290524 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:18.291528 (+ 1004us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:18.292044 (+ 516us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:18.292054 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:18.292551 (+ 497us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:18.293182 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.293198 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.293904 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.293908 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.294967 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.294978 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.295489 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.295495 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.295676 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.296191 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:18.296211 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:18.297930 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:18.299861 (+ 1931us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:18.299868 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:18.299886 (+ 18us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:18.300249 (+ 363us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:18.300594 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:18.300598 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:18.300600 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:18.300787 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:18.301180 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:18.301187 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:18.301501 (+ 314us) client_negotiation.cc:769] Sending connection context
0807 00:05:18.301680 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 00:05:18.302046 (+ 366us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":541,"thread_start_us":132,"threads_started":1}
I20260807 00:05:18.304188 25029 heartbeater.cc:347] Connected to a master server at 127.30.204.126:36415
I20260807 00:05:18.304461 25029 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:18.304994 25029 heartbeater.cc:511] Master 127.30.204.126:36415 requested a full tablet report, sending...
I20260807 00:05:18.306145 24297 ts_manager.cc:194] Registered new tserver with Master: 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235)
I20260807 00:05:18.306756 24297 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:55185
I20260807 00:05:18.312780 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:05:18.317297 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.313507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52526 (local address 127.30.204.126:36415)
0807 00:05:18.313715 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.313720 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.313878 (+ 158us) server_negotiation.cc:407] Connection header received
0807 00:05:18.314070 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.314073 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.314200 (+ 127us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.314302 (+ 102us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:05:18.314563 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.315091 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.316038 (+ 947us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.316271 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.316362 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.316706 (+ 344us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:05:18.316864 (+ 158us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0807 00:05:18.317148 (+ 284us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:52526: token expired
Metrics: {"server-negotiator.queue_time_us":70}
W20260807 00:05:18.317392 24599 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:52526: token expired
W20260807 00:05:18.317796 24185 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.30.204.126:36415)
I20260807 00:05:18.325425 24599 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.318225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52538 (local address 127.30.204.126:36415)
0807 00:05:18.318326 (+ 101us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.318329 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.318342 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:05:18.318470 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.318472 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.318514 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.318577 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:18.319221 (+ 644us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.319791 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.320690 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.320910 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.320990 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.322374 (+ 1384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:18.322377 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:18.322379 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:18.322408 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:18.323805 (+ 1397us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.324196 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.324198 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.324199 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.324274 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:18.324619 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:18.324621 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:18.324622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:18.324827 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:18.324931 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.325196 (+ 265us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.325299 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:05:18.328253 24297 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52538:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:05:18.331933 24297 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:05:18.340879 24498 tablet_service.cc:1467] Processing CreateTablet for tablet feb57ddd0ded4ca6b9206fff86952173 (DEFAULT_TABLE table=test-table [id=789dcf50212a4667a9aa5c46f8508f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:18.341650 24498 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb57ddd0ded4ca6b9206fff86952173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:18.345086 24431 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Bootstrap starting.
I20260807 00:05:18.345757 25044 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.339880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:44529 (local address 127.0.0.1:32898)
0807 00:05:18.340598 (+ 718us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:18.340609 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:18.340704 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:18.341489 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:18.341491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:18.341512 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:18.341626 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.341634 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.342979 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.342983 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.343937 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.343947 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.345316 (+ 1369us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.345320 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.345423 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.345483 (+ 60us) client_negotiation.cc:769] Sending connection context
0807 00:05:18.345558 (+ 75us) client_negotiation.cc:240] Negotiation successful
0807 00:05:18.345630 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":647,"thread_start_us":120,"threads_started":1}
I20260807 00:05:18.346103 24431 tablet_bootstrap.cc:654] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:18.346441 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.340205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32898 (local address 127.30.204.66:44529)
0807 00:05:18.340515 (+ 310us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.340528 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.340691 (+ 163us) server_negotiation.cc:407] Connection header received
0807 00:05:18.340821 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.340827 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.341091 (+ 264us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.341263 (+ 172us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:18.341750 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.342880 (+ 1130us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.344081 (+ 1201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.344976 (+ 895us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.345100 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.346137 (+ 1037us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.346200 (+ 63us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.346299 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":96,"threads_started":1}
I20260807 00:05:18.346699 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.339614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:45235 (local address 127.0.0.1:56384)
0807 00:05:18.339755 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:18.339771 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:18.339854 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:18.342320 (+ 2466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:18.342324 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:18.342344 (+ 20us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:18.342438 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.342444 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.343774 (+ 1330us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.343777 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.344692 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.344702 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.346288 (+ 1586us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.346292 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.346400 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.346428 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 00:05:18.346480 (+ 52us) client_negotiation.cc:240] Negotiation successful
0807 00:05:18.346574 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":60}
I20260807 00:05:18.347383 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.340975 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56384 (local address 127.30.204.67:45235)
0807 00:05:18.341436 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.341450 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.341479 (+ 29us) server_negotiation.cc:407] Connection header received
0807 00:05:18.341579 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.341897 (+ 318us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.342140 (+ 243us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.342325 (+ 185us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:18.342534 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.343650 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.344811 (+ 1161us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.346143 (+ 1332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.346908 (+ 765us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.347086 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.347142 (+ 56us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.347244 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":167,"threads_started":1}
I20260807 00:05:18.348600 24716 tablet_service.cc:1467] Processing CreateTablet for tablet feb57ddd0ded4ca6b9206fff86952173 (DEFAULT_TABLE table=test-table [id=789dcf50212a4667a9aa5c46f8508f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:18.349197 24431 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: No bootstrap required, opened a new log
I20260807 00:05:18.349303 24431 ts_tablet_manager.cc:1397] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.001s
I20260807 00:05:18.349831 24716 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb57ddd0ded4ca6b9206fff86952173. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:18.349820 24431 raft_consensus.cc:348] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.349948 24431 raft_consensus.cc:374] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:18.349990 24431 raft_consensus.cc:729] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743a6966a0454730a1bce6148ab272ac, State: Initialized, Role: FOLLOWER
I20260807 00:05:18.350127 24431 consensus_queue.cc:260] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.350502 24431 ts_tablet_manager.cc:1428] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Time spent starting tablet: real 0.001s user 0.001s sys 0.001s
I20260807 00:05:18.350704 24926 tablet_service.cc:1467] Processing CreateTablet for tablet feb57ddd0ded4ca6b9206fff86952173 (DEFAULT_TABLE table=test-table [id=789dcf50212a4667a9aa5c46f8508f57]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:18.351884 24926 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet feb57ddd0ded4ca6b9206fff86952173. 1 dirs total, 0 dirs full, 0 dirs failed
W20260807 00:05:18.354869 24597 tablet.cc:2369] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:05:18.355289 24640 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Bootstrap starting.
I20260807 00:05:18.357474 24640 tablet_bootstrap.cc:654] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:18.358202 24640 log.cc:824] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:18.359781 25052 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Bootstrap starting.
I20260807 00:05:18.360569 24640 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: No bootstrap required, opened a new log
I20260807 00:05:18.360783 24640 ts_tablet_manager.cc:1397] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:05:18.362661 25052 tablet_bootstrap.cc:654] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:18.363488 25052 log.cc:824] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:18.364378 24640 raft_consensus.cc:348] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.364625 24640 raft_consensus.cc:374] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:18.364689 24640 raft_consensus.cc:729] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 053baa0d2ba14e6baef7f84a8eb38106, State: Initialized, Role: FOLLOWER
I20260807 00:05:18.365126 24640 consensus_queue.cc:260] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.365767 25052 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: No bootstrap required, opened a new log
I20260807 00:05:18.365891 24820 heartbeater.cc:503] Master 127.30.204.126:36415 was elected leader, sending a full tablet report...
I20260807 00:05:18.365991 25052 ts_tablet_manager.cc:1397] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:05:18.366194 24640 ts_tablet_manager.cc:1428] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 00:05:18.369797 25052 raft_consensus.cc:348] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.370103 25052 raft_consensus.cc:374] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:18.370167 25052 raft_consensus.cc:729] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2f089e651c1242cba174dc3fbd5d2ffa, State: Initialized, Role: FOLLOWER
I20260807 00:05:18.370637 25052 consensus_queue.cc:260] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.371394 25029 heartbeater.cc:503] Master 127.30.204.126:36415 was elected leader, sending a full tablet report...
I20260807 00:05:18.371678 25052 ts_tablet_manager.cc:1428] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:05:18.398162 25060 raft_consensus.cc:482] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:18.398409 25060 raft_consensus.cc:504] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.399734 25060 leader_election.cc:288] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529), 743a6966a0454730a1bce6148ab272ac (127.30.204.65:41603)
I20260807 00:05:18.403306 25032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.399956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:44529 (local address 127.30.204.67:52445)
0807 00:05:18.400122 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:18.400137 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:18.400236 (+ 99us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:18.400536 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:18.400538 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:18.400556 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:18.400693 (+ 137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.400699 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.401419 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.401423 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.402253 (+ 830us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.402263 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.402893 (+ 630us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.402897 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.403004 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.403024 (+ 20us) client_negotiation.cc:769] Sending connection context
0807 00:05:18.403088 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 00:05:18.403163 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85,"mutex_wait_us":35}
I20260807 00:05:18.403306 25046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.400189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:52445 (local address 127.30.204.66:44529)
0807 00:05:18.400333 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.400337 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.400351 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:05:18.400400 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.400402 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.400443 (+ 41us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.400505 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:18.400811 (+ 306us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.401320 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.402381 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.402787 (+ 406us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.402893 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.402951 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.403123 (+ 172us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.403206 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:05:18.403539 25061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.399984 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41603 (local address 127.30.204.67:43035)
0807 00:05:18.400395 (+ 411us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:18.400408 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:18.400512 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:18.400790 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:18.400793 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:18.400809 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:18.400923 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.400928 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.401616 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.401619 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.402418 (+ 799us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:18.402427 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.403111 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.403115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:18.403214 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.403236 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:05:18.403295 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:05:18.403363 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"mutex_wait_us":20,"thread_start_us":136,"threads_started":1}
I20260807 00:05:18.403527 25062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.400093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:43035 (local address 127.30.204.65:41603)
0807 00:05:18.400367 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.400372 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.400481 (+ 109us) server_negotiation.cc:407] Connection header received
0807 00:05:18.400619 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.400621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.400677 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.400769 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:18.401026 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.401512 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.402537 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.402991 (+ 454us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.403100 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.403260 (+ 160us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.403308 (+ 48us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.403376 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":201,"thread_start_us":87,"threads_started":1}
I20260807 00:05:18.404052 24749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb57ddd0ded4ca6b9206fff86952173" candidate_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053baa0d2ba14e6baef7f84a8eb38106" is_pre_election: true
I20260807 00:05:18.404070 24527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb57ddd0ded4ca6b9206fff86952173" candidate_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "743a6966a0454730a1bce6148ab272ac" is_pre_election: true
I20260807 00:05:18.404348 24527 raft_consensus.cc:2454] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f089e651c1242cba174dc3fbd5d2ffa in term 0.
I20260807 00:05:18.404361 24749 raft_consensus.cc:2454] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2f089e651c1242cba174dc3fbd5d2ffa in term 0.
I20260807 00:05:18.404829 24854 leader_election.cc:302] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f089e651c1242cba174dc3fbd5d2ffa, 743a6966a0454730a1bce6148ab272ac; no voters:
I20260807 00:05:18.405100 25060 raft_consensus.cc:2781] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:05:18.405195 25060 raft_consensus.cc:482] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:05:18.405247 25060 raft_consensus.cc:3037] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:18.406297 25060 raft_consensus.cc:504] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.406679 25060 leader_election.cc:288] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [CANDIDATE]: Term 1 election: Requested vote from peers 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529), 743a6966a0454730a1bce6148ab272ac (127.30.204.65:41603)
I20260807 00:05:18.407084 24749 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb57ddd0ded4ca6b9206fff86952173" candidate_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "053baa0d2ba14e6baef7f84a8eb38106"
I20260807 00:05:18.407089 24527 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "feb57ddd0ded4ca6b9206fff86952173" candidate_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "743a6966a0454730a1bce6148ab272ac"
I20260807 00:05:18.407218 24527 raft_consensus.cc:3037] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:18.407212 24749 raft_consensus.cc:3037] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:18.408298 24527 raft_consensus.cc:2454] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f089e651c1242cba174dc3fbd5d2ffa in term 1.
I20260807 00:05:18.408562 24749 raft_consensus.cc:2454] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2f089e651c1242cba174dc3fbd5d2ffa in term 1.
I20260807 00:05:18.408593 24854 leader_election.cc:302] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2f089e651c1242cba174dc3fbd5d2ffa, 743a6966a0454730a1bce6148ab272ac; no voters:
I20260807 00:05:18.408890 25060 raft_consensus.cc:2781] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:05:18.409174 25060 raft_consensus.cc:686] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 LEADER]: Becoming Leader. State: Replica: 2f089e651c1242cba174dc3fbd5d2ffa, State: Running, Role: LEADER
I20260807 00:05:18.409548 25060 consensus_queue.cc:237] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:18.412976 24296 catalog_manager.cc:5495] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa reported cstate change: term changed from 0 to 1, leader changed from <none> to 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67). New cstate: current_term: 1 leader_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } health_report { overall_health: UNKNOWN } } }
I20260807 00:05:18.434351 25047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.430592 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56386 (local address 127.30.204.67:45235)
0807 00:05:18.430711 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:18.430715 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:18.430730 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:05:18.430817 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:18.430819 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:18.430876 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:18.430974 (+ 98us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:05:18.431278 (+ 304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.431842 (+ 564us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.432715 (+ 873us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:18.432937 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:18.433021 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:18.433392 (+ 371us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:05:18.433854 (+ 462us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:05:18.433965 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:18.434084 (+ 119us) server_negotiation.cc:299] Negotiation successful
0807 00:05:18.434149 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:05:18.440732 24527 raft_consensus.cc:1261] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Refusing update from remote peer 2f089e651c1242cba174dc3fbd5d2ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:05:18.440733 24749 raft_consensus.cc:1261] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Refusing update from remote peer 2f089e651c1242cba174dc3fbd5d2ffa: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:05:18.441496 25065 consensus_queue.cc:1035] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:05:18.441694 25060 consensus_queue.cc:1035] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [LEADER]: Connected to new peer: Peer: permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:05:18.450773 25070 mvcc.cc:205] Tried to move back new op lower bound from 7315706341124837376 to 7315706341008572416. Current Snapshot: MvccSnapshot[applied={T|T < 7315706341124837376}]
I20260807 00:05:18.457772 24296 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:05:18.457953 24296 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52538:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:05:18.460632 24296 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Sending DeleteTablet for 3 replicas of tablet feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:18.461463 24716 tablet_service.cc:1514] Processing DeleteTablet for tablet feb57ddd0ded4ca6b9206fff86952173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:18 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:32898
I20260807 00:05:18.461457 24926 tablet_service.cc:1514] Processing DeleteTablet for tablet feb57ddd0ded4ca6b9206fff86952173 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:18 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:56384
I20260807 00:05:18.461731 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 24365
I20260807 00:05:18.469524 25082 tablet_replica.cc:330] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: stopping tablet replica
I20260807 00:05:18.469882 25082 raft_consensus.cc:2229] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:05:18.470183 25082 raft_consensus.cc:2258] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:18.470572 25081 tablet_replica.cc:330] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: stopping tablet replica
I20260807 00:05:18.470899 25081 raft_consensus.cc:2229] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:05:18.471308 25081 raft_consensus.cc:2258] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:18.472573 25082 ts_tablet_manager.cc:1905] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:05:18.475853 25082 ts_tablet_manager.cc:1918] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:05:18.475950 25082 log.cc:1196] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:18.476263 25082 ts_tablet_manager.cc:1939] T feb57ddd0ded4ca6b9206fff86952173 P 053baa0d2ba14e6baef7f84a8eb38106: Deleting consensus metadata
I20260807 00:05:18.477784 24273 catalog_manager.cc:4841] TS 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529): tablet feb57ddd0ded4ca6b9206fff86952173 (table test-table [id=789dcf50212a4667a9aa5c46f8508f57]) successfully deleted
I20260807 00:05:18.483594 25081 ts_tablet_manager.cc:1905] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 00:05:18.486613 24272 connection.cc:535] client connection to 127.30.204.65:41603 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:05:18.486807 25081 ts_tablet_manager.cc:1918] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
W20260807 00:05:18.486847 24272 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:05:18.486917 25081 log.cc:1196] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:18.487381 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 24601
I20260807 00:05:18.492453 24604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:18.487772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:41603 (local address 127.0.0.1:35852)
0807 00:05:18.492225 (+ 4453us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:18.492340 (+ 115us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41603: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":4317}
W20260807 00:05:18.500964 24272 catalog_manager.cc:4584] TS 743a6966a0454730a1bce6148ab272ac (127.30.204.65:41603): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet feb57ddd0ded4ca6b9206fff86952173: Network error: Client connection negotiation failed: client connection to 127.30.204.65:41603: connect: Connection refused (error 111)
I20260807 00:05:18.504586 25081 ts_tablet_manager.cc:1939] T feb57ddd0ded4ca6b9206fff86952173 P 2f089e651c1242cba174dc3fbd5d2ffa: Deleting consensus metadata
I20260807 00:05:18.506258 24275 catalog_manager.cc:4841] TS 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235): tablet feb57ddd0ded4ca6b9206fff86952173 (table test-table [id=789dcf50212a4667a9aa5c46f8508f57]) successfully deleted
I20260807 00:05:18.510319 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 24825
I20260807 00:05:18.524014 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 24244
I20260807 00:05:38.540262 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36415
--webserver_interface=127.30.204.126
--webserver_port=43239
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:36415
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:38.657735 25086 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:38.658047 25086 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:38.658099 25086 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:38.662420 25086 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260807 00:05:38.662503 25086 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260807 00:05:38.662518 25086 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260807 00:05:38.662530 25086 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:05:38.662550 25086 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:38.662567 25086 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:38.662582 25086 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:05:38.662596 25086 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:05:38.668485 25086 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:36415
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:36415
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=43239
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25086
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:38.669653 25086 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:38.670756 25086 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:38.680531 25094 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:38.680663 25086 server_base.cc:1034] running on GCE node
W20260807 00:05:38.680578 25098 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:38.680531 25095 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:38.681588 25086 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:38.682838 25086 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:38.684012 25086 hybrid_clock.cc:648] HybridClock initialized: now 1786061138684001 us; error 37 us; skew 500 ppm
Aug 07 00:05:38 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061138, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:38.690928 25086 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:05:38.692157 25086 webserver.cc:460] Webserver started at http://127.30.204.126:43239/ using document root <none> and password file <none>
I20260807 00:05:38.692781 25086 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:38.692863 25086 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:38.697338 25086 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:05:38.699931 25107 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:38.701383 25086 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:38.701553 25086 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "a19425ff05a345668eb4b1729cffe2a0"
format_stamp: "Formatted at 2026-08-07 00:04:55 on dist-test-slave-pww0"
server_key: "6672a34c216625ef5484a239633aeaaf"
server_key_iv: "d44d9a3fdedca6c68b5390c2b13332b9"
server_key_version: "encryptionkey@0"
I20260807 00:05:38.702212 25086 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:38.721015 25086 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:38.724727 25086 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:38.724944 25086 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:38.732664 25086 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:36415
I20260807 00:05:38.732964 25179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:36415 every 8 connection(s)
I20260807 00:05:38.733928 25086 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:05:38.738824 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 25086
I20260807 00:05:38.739003 25180 sys_catalog.cc:263] Verifying existing consensus state
I20260807 00:05:38.739388 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:41603
--local_ip_for_outbound_sockets=127.30.204.65
--tserver_master_addrs=127.30.204.126:36415
--webserver_port=43971
--webserver_interface=127.30.204.65
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:05:38.741549 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Bootstrap starting.
I20260807 00:05:38.760680 25180 log.cc:824] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:38.773506 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:05:38.774039 25180 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Bootstrap complete.
I20260807 00:05:38.778321 25180 raft_consensus.cc:348] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:38.778767 25180 raft_consensus.cc:729] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: a19425ff05a345668eb4b1729cffe2a0, State: Initialized, Role: FOLLOWER
I20260807 00:05:38.779281 25180 consensus_queue.cc:260] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:38.779425 25180 raft_consensus.cc:388] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:05:38.779567 25180 raft_consensus.cc:482] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:05:38.779672 25180 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 2 FOLLOWER]: Advancing to term 3
I20260807 00:05:38.781857 25180 raft_consensus.cc:504] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:38.782231 25180 leader_election.cc:302] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a19425ff05a345668eb4b1729cffe2a0; no voters:
I20260807 00:05:38.782605 25180 leader_election.cc:288] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [CANDIDATE]: Term 3 election: Requested vote from peers
I20260807 00:05:38.782704 25187 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 3 FOLLOWER]: Leader election won for term 3
I20260807 00:05:38.782888 25187 raft_consensus.cc:686] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [term 3 LEADER]: Becoming Leader. State: Replica: a19425ff05a345668eb4b1729cffe2a0, State: Running, Role: LEADER
I20260807 00:05:38.783301 25187 consensus_queue.cc:237] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } }
I20260807 00:05:38.784040 25180 sys_catalog.cc:564] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:05:38.784783 25189 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "a19425ff05a345668eb4b1729cffe2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } } }
I20260807 00:05:38.784914 25189 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:38.785717 25192 sys_catalog.cc:455] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a19425ff05a345668eb4b1729cffe2a0. Latest consensus state: current_term: 3 leader_uuid: "a19425ff05a345668eb4b1729cffe2a0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a19425ff05a345668eb4b1729cffe2a0" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 36415 } } }
I20260807 00:05:38.785821 25192 sys_catalog.cc:458] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:38.785877 25198 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:05:38.791734 25198 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:05:38.792611 25198 catalog_manager.cc:1259] Loaded cluster ID: fe43b79711d04b5c84ee192046b002e1
I20260807 00:05:38.792714 25198 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:05:38.795354 25198 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:05:38.796295 25198 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Loaded TSK: 0
I20260807 00:05:38.796401 25198 catalog_manager.cc:5879] T 00000000000000000000000000000000 P a19425ff05a345668eb4b1729cffe2a0: Loaded TSK: 1
F20260807 00:05:38.797251 25198 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0a9f9a29ad google::LogMessage::Fail() at ??:0
@ 0x7f0a9f9a588a google::LogMessage::SendToLog() at ??:0
@ 0x7f0a9f9a24de google::LogMessage::Flush() at ??:0
@ 0x7f0a9f9a3a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0aa266618d kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0aa265ef0d kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0aaa30b7bb kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0aaa2e599c kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0aaa2e6178 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlvE3_clEv at ??:0
@ 0x7f0aaa327df8 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f0aaa34682b std::function<>::operator()() at ??:0
@ 0x7f0aaa2e5b4f _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_ at ??:0
@ 0x7f0aaa2e6f3e kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0aaa2e3579 _ZZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvENKUlvE_clEv at ??:0
@ 0x7f0aaa327640 _ZNSt17_Function_handlerIFvvEZN4kudu6master14CatalogManager17ElectedAsLeaderCbEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0aa1bf43d7 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0aa1bf4c83 _ZZN4kudu10ThreadPool12CreateThreadEvENKUlvE_clEv at ??:0
@ 0x7f0aa1bf63f2 _ZNSt17_Function_handlerIFvvEZN4kudu10ThreadPool12CreateThreadEvEUlvE_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x61d838 std::function<>::operator()() at ??:0
@ 0x7f0aa1be5512 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0aa32296db start_thread at ??:0
@ 0x7f0a9e4a061f clone at ??:0
W20260807 00:05:38.893601 25184 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:38.894026 25184 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:38.894104 25184 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:38.900383 25184 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:38.900493 25184 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:38.900584 25184 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:05:38.909050 25184 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:41603
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=43971
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25184
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:38.910758 25184 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:38.912320 25184 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:38.926126 25268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:38.926163 25265 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:38.926126 25264 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:38.926527 25184 server_base.cc:1034] running on GCE node
I20260807 00:05:38.927003 25184 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:38.928097 25184 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:38.929303 25184 hybrid_clock.cc:648] HybridClock initialized: now 1786061138929265 us; error 58 us; skew 500 ppm
Aug 07 00:05:38 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061138, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:38.932446 25184 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:05:38.933522 25184 webserver.cc:460] Webserver started at http://127.30.204.65:43971/ using document root <none> and password file <none>
I20260807 00:05:38.934140 25184 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:38.934214 25184 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:38.938956 25184 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:05:38.941778 25281 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:38.943383 25184 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:05:38.943601 25184 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "743a6966a0454730a1bce6148ab272ac"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "b6fea95a4cfdb65006cf0464370210dc"
server_key_iv: "53d4810c9cecb98383a0cedd1f38f50e"
server_key_version: "encryptionkey@0"
I20260807 00:05:38.944219 25184 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:38.957434 25184 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:38.961525 25184 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:38.961760 25184 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:38.962471 25184 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:38.963830 25295 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260807 00:05:38.969039 25184 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260807 00:05:38.969210 25184 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.006s user 0.000s sys 0.000s
I20260807 00:05:38.969357 25184 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260807 00:05:38.971673 25184 ts_tablet_manager.cc:610] Registered 1 tablets
I20260807 00:05:38.971771 25184 ts_tablet_manager.cc:589] Time spent register tablets: real 0.002s user 0.003s sys 0.000s
I20260807 00:05:38.972093 25295 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Bootstrap starting.
I20260807 00:05:38.984295 25184 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:41603
I20260807 00:05:38.985891 25184 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:05:38.986977 25481 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:41603 every 8 connection(s)
I20260807 00:05:38.988999 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 25184
I20260807 00:05:38.989532 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:44529
--local_ip_for_outbound_sockets=127.30.204.66
--tserver_master_addrs=127.30.204.126:36415
--webserver_port=34273
--webserver_interface=127.30.204.66
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:05:38.997267 25295 log.cc:824] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:39.005208 25295 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260807 00:05:39.005642 25295 tablet_bootstrap.cc:492] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Bootstrap complete.
I20260807 00:05:39.006760 25295 ts_tablet_manager.cc:1397] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Time spent bootstrapping tablet: real 0.035s user 0.025s sys 0.004s
I20260807 00:05:39.009479 25295 raft_consensus.cc:348] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:39.009971 25295 raft_consensus.cc:729] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 743a6966a0454730a1bce6148ab272ac, State: Initialized, Role: FOLLOWER
I20260807 00:05:39.010529 25295 consensus_queue.cc:260] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:39.011644 25295 ts_tablet_manager.cc:1428] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac: Time spent starting tablet: real 0.005s user 0.008s sys 0.000s
W20260807 00:05:39.066495 25485 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:05:38.988577 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:33503)
0807 00:05:38.989181 (+ 604us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:38.989245 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:38.990433 (+ 1188us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:39.065288 (+ 74855us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":358,"thread_start_us":158,"threads_started":1}
W20260807 00:05:39.067423 25483 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:36415 (0 consecutive failures): Network error: Failed to ping master at 127.30.204.126:36415: Client connection negotiation failed: client connection to 127.30.204.126:36415: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260807 00:05:39.068558 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.068173 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:55259)
0807 00:05:39.068372 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.068472 (+ 100us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 00:05:39.069410 25485 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.069156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:60217)
0807 00:05:39.069281 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.069340 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260807 00:05:39.069600 25483 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260807 00:05:39.133123 25486 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:39.133584 25486 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:39.133675 25486 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:39.138460 25486 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:39.138553 25486 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:39.138617 25486 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:05:39.144765 25486 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:44529
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=34273
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25486
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:39.146005 25486 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:39.147167 25486 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:39.158105 25498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:39.158175 25486 server_base.cc:1034] running on GCE node
W20260807 00:05:39.158142 25502 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:39.158109 25496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:39.158943 25486 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:39.159953 25486 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:39.161154 25486 hybrid_clock.cc:648] HybridClock initialized: now 1786061139161126 us; error 40 us; skew 500 ppm
Aug 07 00:05:39 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061139, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:39.164532 25486 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:05:39.165589 25486 webserver.cc:460] Webserver started at http://127.30.204.66:34273/ using document root <none> and password file <none>
I20260807 00:05:39.166244 25486 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:39.166328 25486 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:39.170831 25486 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:05:39.173442 25513 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:39.174955 25486 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:05:39.175097 25486 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "053baa0d2ba14e6baef7f84a8eb38106"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "25deb440e738f0fbc1b294dcf09ff545"
server_key_iv: "86907273a3f2b459bdc3fb87147f9ba7"
server_key_version: "encryptionkey@0"
I20260807 00:05:39.175788 25486 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:39.196204 25486 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:39.199681 25486 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:39.199898 25486 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:39.200582 25486 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:39.201526 25486 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:39.201578 25486 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:39.201638 25486 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:39.201668 25486 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:39.211918 25486 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:44529
I20260807 00:05:39.211948 25699 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:44529 every 8 connection(s)
I20260807 00:05:39.213085 25486 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:05:39.217535 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.215638 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:33413)
0807 00:05:39.216233 (+ 595us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.216714 (+ 481us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":341,"thread_start_us":138,"threads_started":1}
W20260807 00:05:39.218123 25700 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:36415 (0 consecutive failures): Network error: Failed to ping master at 127.30.204.126:36415: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
I20260807 00:05:39.218822 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.218566 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:34337)
0807 00:05:39.218696 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.218755 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
I20260807 00:05:39.219741 25702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.219405 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:46283)
0807 00:05:39.219608 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.219674 (+ 66us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120}
W20260807 00:05:39.219969 25700 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:05:39.220502 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 25486
I20260807 00:05:39.220942 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:45235
--local_ip_for_outbound_sockets=127.30.204.67
--tserver_master_addrs=127.30.204.126:36415
--webserver_port=35669
--webserver_interface=127.30.204.67
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38139
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:39.343504 25704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:39.343818 25704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:39.343864 25704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:39.348551 25704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:39.348639 25704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:39.348703 25704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:05:39.354770 25704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38139
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:45235
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=35669
--tserver_master_addrs=127.30.204.126:36415
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:39.356019 25704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:39.357149 25704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:39.367445 25715 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:39.367488 25713 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:39.367488 25717 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:39.367741 25704 server_base.cc:1034] running on GCE node
I20260807 00:05:39.368305 25704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:39.369071 25704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:39.370229 25704 hybrid_clock.cc:648] HybridClock initialized: now 1786061139370203 us; error 43 us; skew 500 ppm
Aug 07 00:05:39 dist-test-slave-pww0 krb5kdc[23366](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061139, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:39.373209 25704 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:05:39.374262 25704 webserver.cc:460] Webserver started at http://127.30.204.67:35669/ using document root <none> and password file <none>
I20260807 00:05:39.374861 25704 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:39.374933 25704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:39.379317 25704 fs_manager.cc:709] Time spent opening directory manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:39.381955 25728 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:39.383637 25704 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:39.383777 25704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "2f089e651c1242cba174dc3fbd5d2ffa"
format_stamp: "Formatted at 2026-08-07 00:04:56 on dist-test-slave-pww0"
server_key: "5e69a0469bb2ed694ecec407f3314514"
server_key_iv: "b4ce46c8e851d215c5706fecda0fb32e"
server_key_version: "encryptionkey@0"
I20260807 00:05:39.384408 25704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:39.399348 25704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:39.402781 25704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:39.402988 25704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:39.403704 25704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:39.404651 25704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:39.404701 25704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:39.404758 25704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:39.404788 25704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:39.414448 25704 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:45235
I20260807 00:05:39.414466 25913 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:45235 every 8 connection(s)
I20260807 00:05:39.415632 25704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:05:39.419970 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.418101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:51679)
0807 00:05:39.418905 (+ 804us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.419038 (+ 133us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":557,"thread_start_us":143,"threads_started":1}
I20260807 00:05:39.420126 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 25704
W20260807 00:05:39.420557 25914 heartbeater.cc:644] Failed to heartbeat to 127.30.204.126:36415 (0 consecutive failures): Network error: Failed to ping master at 127.30.204.126:36415: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
I20260807 00:05:39.421342 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.421026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:39325)
0807 00:05:39.421191 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.421260 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":56}
I20260807 00:05:39.422021 25916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:39.421807 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:50683)
0807 00:05:39.421917 (+ 110us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:39.421966 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 00:05:39.422197 25914 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260807 00:05:40.071004 25919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:40.070510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:50245)
0807 00:05:40.070822 (+ 312us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:40.070884 (+ 62us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":204,"thread_start_us":82,"threads_started":1}
I20260807 00:05:40.191365 25921 raft_consensus.cc:482] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:40.191625 25921 raft_consensus.cc:504] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:40.192802 25921 leader_election.cc:288] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235), 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
Aug 07 00:05:40 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061138, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.67@KRBTEST.COM
Aug 07 00:05:40 dist-test-slave-pww0 krb5kdc[23366](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061138, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.66@KRBTEST.COM
I20260807 00:05:40.207868 25922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:40.193102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:45235 (local address 127.30.204.65:44221)
0807 00:05:40.193778 (+ 676us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:40.193788 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:40.193879 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:40.195442 (+ 1563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:40.195449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:40.196082 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:40.196526 (+ 444us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:40.196542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.197887 (+ 1345us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.197890 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:40.198867 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:40.198878 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.199695 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.199699 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:40.199856 (+ 157us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:40.200502 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:40.200521 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:40.202251 (+ 1730us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:40.204928 (+ 2677us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:40.204936 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:40.204953 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:40.205425 (+ 472us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:40.205834 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:40.205838 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:40.205840 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:40.206018 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:40.206756 (+ 738us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:40.206763 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:40.207035 (+ 272us) client_negotiation.cc:769] Sending connection context
0807 00:05:40.207215 (+ 180us) client_negotiation.cc:240] Negotiation successful
0807 00:05:40.207654 (+ 439us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":620,"mutex_wait_us":277,"spinlock_wait_cycles":57472,"thread_start_us":69,"threads_started":1}
I20260807 00:05:40.207886 25923 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:40.193287 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:44221 (local address 127.30.204.67:45235)
0807 00:05:40.193623 (+ 336us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:40.193641 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:40.193893 (+ 252us) server_negotiation.cc:407] Connection header received
0807 00:05:40.194617 (+ 724us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:40.194623 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:40.195006 (+ 383us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:40.195325 (+ 319us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:40.196736 (+ 1411us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.197722 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.199023 (+ 1301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.199560 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.199726 (+ 166us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:40.202525 (+ 2799us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:40.202532 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:40.202546 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:40.202582 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:40.204768 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:40.205566 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:40.205579 (+ 13us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:40.205583 (+ 4us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:40.205721 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:40.206126 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:40.206129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:40.206131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:40.206623 (+ 492us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:40.206755 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:40.207378 (+ 623us) server_negotiation.cc:299] Negotiation successful
0807 00:05:40.207707 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":74,"threads_started":1}
I20260807 00:05:40.207899 25919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:40.193101 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:44529 (local address 127.30.204.65:49127)
0807 00:05:40.193476 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:40.193493 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:40.193607 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:40.195442 (+ 1835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:40.195449 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:40.196033 (+ 584us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:40.196526 (+ 493us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:40.196542 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.197884 (+ 1342us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.197887 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:40.198864 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:40.198874 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.199635 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.199638 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:40.199850 (+ 212us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:40.200556 (+ 706us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:40.200569 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:40.202564 (+ 1995us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:40.204928 (+ 2364us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:40.204936 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:40.204953 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:40.205431 (+ 478us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:40.205840 (+ 409us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:40.205842 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:40.205844 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:40.206018 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:40.206772 (+ 754us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:40.206775 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:40.207034 (+ 259us) client_negotiation.cc:769] Sending connection context
0807 00:05:40.207215 (+ 181us) client_negotiation.cc:240] Negotiation successful
0807 00:05:40.207745 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"mutex_wait_us":308,"spinlock_wait_cycles":32128}
I20260807 00:05:40.207914 25924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:40.193391 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:49127 (local address 127.30.204.66:44529)
0807 00:05:40.194042 (+ 651us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:40.194056 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:40.194094 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:05:40.194740 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:40.194745 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:40.195057 (+ 312us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:40.195326 (+ 269us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:40.196735 (+ 1409us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.197730 (+ 995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.199003 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:40.199508 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:40.199671 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:40.202795 (+ 3124us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:40.202802 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:40.202819 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:40.202859 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:40.204768 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:40.205566 (+ 798us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:40.205572 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:40.205577 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:40.205722 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:40.206126 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:40.206129 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:40.206131 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:40.206617 (+ 486us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:40.206756 (+ 139us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:40.207380 (+ 624us) server_negotiation.cc:299] Negotiation successful
0807 00:05:40.207705 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":257,"thread_start_us":90,"threads_started":1}
W20260807 00:05:40.210876 25290 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
W20260807 00:05:40.211549 25284 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:40.211679 25284 leader_election.cc:302] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 743a6966a0454730a1bce6148ab272ac; no voters: 053baa0d2ba14e6baef7f84a8eb38106, 2f089e651c1242cba174dc3fbd5d2ffa
I20260807 00:05:40.212157 25921 raft_consensus.cc:2726] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:05:40.221153 25925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:40.220643 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:57287)
0807 00:05:40.220973 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:40.221040 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":236,"thread_start_us":100,"threads_started":1}
I20260807 00:05:40.423395 25926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:40.422909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:39881)
0807 00:05:40.423193 (+ 284us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:40.423283 (+ 90us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":198,"thread_start_us":81,"threads_started":1}
I20260807 00:05:41.072433 25927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:41.071949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:58423)
0807 00:05:41.072256 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:41.072326 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":86,"threads_started":1}
I20260807 00:05:41.222726 25931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:41.222226 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:47175)
0807 00:05:41.222522 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:41.222583 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":217,"thread_start_us":82,"threads_started":1}
I20260807 00:05:41.424803 25934 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:41.424309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:54035)
0807 00:05:41.424618 (+ 309us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:41.424682 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":82,"threads_started":1}
I20260807 00:05:42.073933 25936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:42.073450 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:60387)
0807 00:05:42.073756 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:42.073828 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":97,"threads_started":1}
I20260807 00:05:42.224255 25938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:42.223687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:48181)
0807 00:05:42.224067 (+ 380us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:42.224144 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":90,"threads_started":1}
I20260807 00:05:42.255820 25939 raft_consensus.cc:482] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:42.255961 25939 raft_consensus.cc:504] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:42.256446 25939 leader_election.cc:288] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235), 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
W20260807 00:05:42.257347 25290 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
W20260807 00:05:42.257485 25284 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:42.257578 25284 leader_election.cc:302] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 743a6966a0454730a1bce6148ab272ac; no voters: 053baa0d2ba14e6baef7f84a8eb38106, 2f089e651c1242cba174dc3fbd5d2ffa
I20260807 00:05:42.257746 25939 raft_consensus.cc:2726] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:05:42.426321 25941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:42.425777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:48555)
0807 00:05:42.426140 (+ 363us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:42.426211 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":97,"threads_started":1}
I20260807 00:05:43.075246 25942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:43.074747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:59645)
0807 00:05:43.075043 (+ 296us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:43.075111 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":83,"threads_started":1}
I20260807 00:05:43.225708 25945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:43.225187 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:57787)
0807 00:05:43.225528 (+ 341us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:43.225601 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":83,"threads_started":1}
I20260807 00:05:43.427903 25946 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:43.427301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:37601)
0807 00:05:43.427707 (+ 406us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:43.427786 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":305,"thread_start_us":107,"threads_started":1}
I20260807 00:05:44.076948 25948 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:44.076368 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:45739)
0807 00:05:44.076766 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:44.076840 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":278,"thread_start_us":81,"threads_started":1}
I20260807 00:05:44.227339 25949 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:44.226787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:58087)
0807 00:05:44.227133 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:44.227216 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":95,"threads_started":1}
I20260807 00:05:44.429486 25952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:44.428916 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:40279)
0807 00:05:44.429299 (+ 383us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:44.429375 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":279,"thread_start_us":91,"threads_started":1}
I20260807 00:05:45.078394 25953 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:45.077913 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:36109)
0807 00:05:45.078219 (+ 306us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:45.078290 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":87,"threads_started":1}
I20260807 00:05:45.228835 25954 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:45.228274 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:38025)
0807 00:05:45.228650 (+ 376us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:45.228726 (+ 76us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":256,"thread_start_us":86,"threads_started":1}
I20260807 00:05:45.431042 25956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:45.430463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:42897)
0807 00:05:45.430864 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:45.430937 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":304,"thread_start_us":129,"threads_started":1}
I20260807 00:05:45.479264 25957 raft_consensus.cc:482] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:45.479406 25957 raft_consensus.cc:504] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:45.479926 25957 leader_election.cc:288] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235), 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
W20260807 00:05:45.480786 25290 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
W20260807 00:05:45.480913 25284 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:45.480975 25284 leader_election.cc:302] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 743a6966a0454730a1bce6148ab272ac; no voters: 053baa0d2ba14e6baef7f84a8eb38106, 2f089e651c1242cba174dc3fbd5d2ffa
I20260807 00:05:45.481138 25957 raft_consensus.cc:2726] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:05:46.079895 25958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:46.079388 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:48305)
0807 00:05:46.079723 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:46.079793 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":248,"thread_start_us":131,"threads_started":1}
I20260807 00:05:46.230278 25959 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:46.229754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:50271)
0807 00:05:46.230093 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:46.230160 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":80,"threads_started":1}
I20260807 00:05:46.432539 25961 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:46.431974 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:50951)
0807 00:05:46.432351 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:46.432425 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":286,"thread_start_us":80,"threads_started":1}
I20260807 00:05:47.081250 25963 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:47.080776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:60853)
0807 00:05:47.081058 (+ 282us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:47.081144 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":199,"thread_start_us":77,"threads_started":1}
I20260807 00:05:47.231670 25964 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:47.231174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:33927)
0807 00:05:47.231496 (+ 322us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:47.231565 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":207,"thread_start_us":86,"threads_started":1}
I20260807 00:05:47.283530 25967 raft_consensus.cc:482] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:47.283689 25967 raft_consensus.cc:504] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:47.284163 25967 leader_election.cc:288] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235), 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
W20260807 00:05:47.285038 25284 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
W20260807 00:05:47.285189 25290 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:47.285292 25290 leader_election.cc:302] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 743a6966a0454730a1bce6148ab272ac; no voters: 053baa0d2ba14e6baef7f84a8eb38106, 2f089e651c1242cba174dc3fbd5d2ffa
I20260807 00:05:47.285465 25967 raft_consensus.cc:2726] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:05:47.434099 25969 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:47.433582 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:41885)
0807 00:05:47.433924 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:47.433994 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":244,"thread_start_us":105,"threads_started":1}
I20260807 00:05:48.082729 25970 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:48.082207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:52999)
0807 00:05:48.082526 (+ 319us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:48.082598 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":84,"threads_started":1}
I20260807 00:05:48.233008 25971 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:48.232490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:54599)
0807 00:05:48.232832 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:48.232900 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":128,"threads_started":1}
I20260807 00:05:48.435563 25972 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:48.434978 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:33219)
0807 00:05:48.435320 (+ 342us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:48.435403 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":240,"thread_start_us":99,"threads_started":1}
I20260807 00:05:49.079358 25973 raft_consensus.cc:482] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:49.079547 25973 raft_consensus.cc:504] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2f089e651c1242cba174dc3fbd5d2ffa" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45235 } } peers { permanent_uuid: "053baa0d2ba14e6baef7f84a8eb38106" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 44529 } } peers { permanent_uuid: "743a6966a0454730a1bce6148ab272ac" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 41603 } }
I20260807 00:05:49.080031 25973 leader_election.cc:288] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235), 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529)
W20260807 00:05:49.080950 25284 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 053baa0d2ba14e6baef7f84a8eb38106 (127.30.204.66:44529): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
W20260807 00:05:49.081079 25290 leader_election.cc:341] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 2f089e651c1242cba174dc3fbd5d2ffa (127.30.204.67:45235): Not found: Tablet not found: feb57ddd0ded4ca6b9206fff86952173
I20260807 00:05:49.081151 25290 leader_election.cc:302] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 743a6966a0454730a1bce6148ab272ac; no voters: 053baa0d2ba14e6baef7f84a8eb38106, 2f089e651c1242cba174dc3fbd5d2ffa
I20260807 00:05:49.081342 25973 raft_consensus.cc:2726] T feb57ddd0ded4ca6b9206fff86952173 P 743a6966a0454730a1bce6148ab272ac [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260807 00:05:49.084112 25977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:49.083665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:47315)
0807 00:05:49.083940 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:49.084009 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":75,"threads_started":1}
I20260807 00:05:49.234531 25978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:49.233986 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:44379)
0807 00:05:49.234332 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:49.234417 (+ 85us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":239,"thread_start_us":83,"threads_started":1}
I20260807 00:05:49.437071 25980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:49.436549 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:52595)
0807 00:05:49.436885 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:49.436959 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":243,"thread_start_us":98,"threads_started":1}
I20260807 00:05:50.085492 25981 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:50.084975 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:40199)
0807 00:05:50.085304 (+ 329us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:50.085377 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":104,"threads_started":1}
I20260807 00:05:50.236088 25983 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:50.235550 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:60529)
0807 00:05:50.235908 (+ 358us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:50.235980 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":260,"thread_start_us":124,"threads_started":1}
I20260807 00:05:50.438556 25986 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:50.438038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:59719)
0807 00:05:50.438378 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:50.438452 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":250,"thread_start_us":117,"threads_started":1}
I20260807 00:05:51.086988 25989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:51.086445 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:53079)
0807 00:05:51.086804 (+ 359us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:51.086875 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":99,"threads_started":1}
I20260807 00:05:51.237681 25990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:51.237110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:52763)
0807 00:05:51.237504 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:51.237577 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":103,"threads_started":1}
I20260807 00:05:51.440147 25991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:51.439596 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:57587)
0807 00:05:51.439962 (+ 366us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:51.440037 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":108,"threads_started":1}
I20260807 00:05:52.088435 25992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:52.087858 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:57505)
0807 00:05:52.088235 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:52.088321 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":255,"thread_start_us":101,"threads_started":1}
I20260807 00:05:52.239226 25993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:52.238689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:35489)
0807 00:05:52.239037 (+ 348us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:52.239120 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":252,"thread_start_us":107,"threads_started":1}
I20260807 00:05:52.441759 25996 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:52.441202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:39717)
0807 00:05:52.441579 (+ 377us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:52.441654 (+ 75us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":282,"thread_start_us":101,"threads_started":1}
I20260807 00:05:53.089797 25997 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:53.089287 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:45415)
0807 00:05:53.089626 (+ 339us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:53.089693 (+ 67us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":224,"thread_start_us":86,"threads_started":1}
I20260807 00:05:53.240705 25999 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:53.240184 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:43335)
0807 00:05:53.240492 (+ 308us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:53.240573 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":216,"thread_start_us":89,"threads_started":1}
I20260807 00:05:53.443216 26001 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:53.442727 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:35017)
0807 00:05:53.443034 (+ 307us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:53.443112 (+ 78us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":90,"threads_started":1}
I20260807 00:05:54.091377 26007 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:54.090905 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.65:38979)
0807 00:05:54.091203 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:54.091273 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":79,"threads_started":1}
I20260807 00:05:54.242209 26008 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:54.241686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.66:51875)
0807 00:05:54.242032 (+ 346us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:54.242106 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":251,"thread_start_us":82,"threads_started":1}
I20260807 00:05:54.425768 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 25184
I20260807 00:05:54.437175 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 25486
I20260807 00:05:54.449431 26009 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:54.445591 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:36415 (local address 127.30.204.67:53373)
0807 00:05:54.449223 (+ 3632us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:54.449309 (+ 86us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.126:36415: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":3533,"thread_start_us":3220,"threads_started":1}
I20260807 00:05:54.454730 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 25704
2026-08-07T00:05:54Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (60048 ms)
[----------] 30 tests from SecurityITest (253038 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:05:54 dist-test-slave-pww0 krb5kdc[26012](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:05:54 dist-test-slave-pww0 krb5kdc[26012](info): set up 2 sockets
Aug 07 00:05:54 dist-test-slave-pww0 krb5kdc[26012](info): commencing operation
krb5kdc: starting...
W20260807 00:05:55.647287 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:05:55 dist-test-slave-pww0 krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:05:55Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:05:55Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:05:55.786725 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37363
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:37363
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:55.909736 26030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:55.910038 26030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:55.910081 26030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:55.914312 26030 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:05:55.914391 26030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:55.914412 26030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:55.914428 26030 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:05:55.914443 26030 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:05:55.920239 26030 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:37363
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37363
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:55.921442 26030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:55.922536 26030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:55.931885 26039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:55.931874 26042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:55.931982 26030 server_base.cc:1034] running on GCE node
W20260807 00:05:55.931874 26040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:55.932837 26030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:55.934123 26030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:55.935350 26030 hybrid_clock.cc:648] HybridClock initialized: now 1786061155935322 us; error 46 us; skew 500 ppm
Aug 07 00:05:55 dist-test-slave-pww0 krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061155, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:55.938370 26030 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:05:55.939388 26030 webserver.cc:460] Webserver started at http://127.30.204.126:44731/ using document root <none> and password file <none>
I20260807 00:05:55.940006 26030 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:55.940073 26030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:55.940260 26030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:55.942490 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "afcd0bfc4b8d4076b595c8b2994dc378"
format_stamp: "Formatted at 2026-08-07 00:05:55 on dist-test-slave-pww0"
server_key: "9fce6a8a8cd10b4b298d4e94bd76ba02"
server_key_iv: "6f4efea68dadc630f3ce75471219115b"
server_key_version: "encryptionkey@0"
I20260807 00:05:55.943092 26030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "afcd0bfc4b8d4076b595c8b2994dc378"
format_stamp: "Formatted at 2026-08-07 00:05:55 on dist-test-slave-pww0"
server_key: "9fce6a8a8cd10b4b298d4e94bd76ba02"
server_key_iv: "6f4efea68dadc630f3ce75471219115b"
server_key_version: "encryptionkey@0"
I20260807 00:05:55.947291 26030 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 00:05:55.950194 26051 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:55.951784 26030 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:55.951920 26030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "afcd0bfc4b8d4076b595c8b2994dc378"
format_stamp: "Formatted at 2026-08-07 00:05:55 on dist-test-slave-pww0"
server_key: "9fce6a8a8cd10b4b298d4e94bd76ba02"
server_key_iv: "6f4efea68dadc630f3ce75471219115b"
server_key_version: "encryptionkey@0"
I20260807 00:05:55.952037 26030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:55.969426 26030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:55.973237 26030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:55.973423 26030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:55.980890 26030 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:37363
I20260807 00:05:55.981007 26142 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:37363 every 8 connection(s)
I20260807 00:05:55.982024 26030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:05:55.985296 26145 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:55.985445 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 26030
I20260807 00:05:55.985575 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:05:55.985860 31537 external_mini_cluster.cc:1428] Setting key b5e440a0a6fb216103a764be975c9028
I20260807 00:05:55.993285 26145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378: Bootstrap starting.
Aug 07 00:05:55 dist-test-slave-pww0 krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061155, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:55.996011 26145 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:55.996927 26145 log.cc:824] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:55.999596 26145 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378: No bootstrap required, opened a new log
I20260807 00:05:56.002329 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:55.987505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52934 (local address 127.30.204.126:37363)
0807 00:05:55.988008 (+ 503us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:55.988030 (+ 22us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:55.988079 (+ 49us) server_negotiation.cc:407] Connection header received
0807 00:05:55.988907 (+ 828us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:55.988938 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:55.989427 (+ 489us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:55.989848 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:55.990641 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:55.992236 (+ 1595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:55.993201 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:55.993910 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:55.994080 (+ 170us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:55.996642 (+ 2562us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:55.996648 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:55.996669 (+ 21us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:55.996720 (+ 51us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:55.998700 (+ 1980us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:55.999204 (+ 504us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:55.999211 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:55.999217 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:55.999372 (+ 155us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:55.999744 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:55.999747 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:55.999748 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.000306 (+ 558us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:56.000516 (+ 210us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.000837 (+ 321us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.001183 (+ 346us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":144,"threads_started":1}
I20260807 00:05:56.004400 26145 raft_consensus.cc:348] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37363 } }
I20260807 00:05:56.004670 26145 raft_consensus.cc:374] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:56.004706 26145 raft_consensus.cc:729] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: afcd0bfc4b8d4076b595c8b2994dc378, State: Initialized, Role: FOLLOWER
I20260807 00:05:56.005246 26145 consensus_queue.cc:260] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37363 } }
I20260807 00:05:56.005401 26145 raft_consensus.cc:388] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:05:56.005446 26145 raft_consensus.cc:482] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:05:56.005525 26145 raft_consensus.cc:3037] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:56.006606 26145 raft_consensus.cc:504] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37363 } }
I20260807 00:05:56.007030 26145 leader_election.cc:302] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: afcd0bfc4b8d4076b595c8b2994dc378; no voters:
I20260807 00:05:56.007483 26145 leader_election.cc:288] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:05:56.007588 26154 raft_consensus.cc:2781] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:05:56.008020 26154 raft_consensus.cc:686] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [term 1 LEADER]: Becoming Leader. State: Replica: afcd0bfc4b8d4076b595c8b2994dc378, State: Running, Role: LEADER
I20260807 00:05:56.008436 26154 consensus_queue.cc:237] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37363 } }
I20260807 00:05:56.009107 26145 sys_catalog.cc:564] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:05:56.010408 26156 sys_catalog.cc:455] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [sys.catalog]: SysCatalogTable state changed. Reason: New leader afcd0bfc4b8d4076b595c8b2994dc378. Latest consensus state: current_term: 1 leader_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37363 } } }
I20260807 00:05:56.010615 26156 sys_catalog.cc:458] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:56.010412 26158 sys_catalog.cc:455] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "afcd0bfc4b8d4076b595c8b2994dc378" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37363 } } }
I20260807 00:05:56.011305 26158 sys_catalog.cc:458] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:56.011384 26166 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:05:56.014530 26166 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:05:56.020952 26166 catalog_manager.cc:1347] Generated new cluster ID: 1fa204dae2294d4e99faf97916ea7162
I20260807 00:05:56.021046 26166 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:05:56.034183 26166 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:05:56.035687 26166 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:05:56.055431 26166 catalog_manager.cc:5868] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378: Generated new TSK 0
I20260807 00:05:56.056480 26166 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:05:56.094455 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37363
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:56.216557 26186 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:56.216876 26186 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:56.216922 26186 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:56.221251 26186 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:56.221330 26186 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:56.221395 26186 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:05:56.226977 26186 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26186
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:56.228202 26186 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:56.229281 26186 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:56.240042 26196 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:56.240046 26194 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:56.240105 26186 server_base.cc:1034] running on GCE node
W20260807 00:05:56.240042 26199 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:56.240839 26186 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:56.241549 26186 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:56.242722 26186 hybrid_clock.cc:648] HybridClock initialized: now 1786061156242694 us; error 42 us; skew 500 ppm
Aug 07 00:05:56 dist-test-slave-pww0 krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061156, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:56.245714 26186 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:05:56.246735 26186 webserver.cc:460] Webserver started at http://127.30.204.65:45863/ using document root <none> and password file <none>
I20260807 00:05:56.247349 26186 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:56.247424 26186 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:56.247681 26186 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:56.249980 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "2fd0c19e3949458fa1416da81e80cb12"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "fc50ab53a82c02ba3ca8724dda20378d"
server_key_iv: "2ef4ba8bc6f9a622714bd79a9ea5d1e8"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.250640 26186 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "2fd0c19e3949458fa1416da81e80cb12"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "fc50ab53a82c02ba3ca8724dda20378d"
server_key_iv: "2ef4ba8bc6f9a622714bd79a9ea5d1e8"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.255774 26186 fs_manager.cc:691] Time spent creating directory manager: real 0.005s user 0.007s sys 0.000s
I20260807 00:05:56.258985 26211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.260615 26186 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:56.260774 26186 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "2fd0c19e3949458fa1416da81e80cb12"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "fc50ab53a82c02ba3ca8724dda20378d"
server_key_iv: "2ef4ba8bc6f9a622714bd79a9ea5d1e8"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.260905 26186 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:56.280674 26186 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:56.284349 26186 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:56.284562 26186 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:56.285259 26186 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:56.286213 26186 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:56.286265 26186 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.286324 26186 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:56.286353 26186 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.296142 26186 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:40649
I20260807 00:05:56.296169 26404 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:40649 every 8 connection(s)
I20260807 00:05:56.297166 26186 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:05:56.302858 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 26186
I20260807 00:05:56.303004 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:05:56.303359 31537 external_mini_cluster.cc:1428] Setting key d67a81798206289016825867f00a1da7
Aug 07 00:05:56 dist-test-slave-pww0 krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061156, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:56.313138 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.299166 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:38851 (local address 127.30.204.126:37363)
0807 00:05:56.299321 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:56.299325 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:56.300265 (+ 940us) server_negotiation.cc:407] Connection header received
0807 00:05:56.301648 (+ 1383us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:56.301652 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:56.301714 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:56.301788 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:56.303271 (+ 1483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.304002 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.305571 (+ 1569us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.305830 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.305925 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.308838 (+ 2913us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:56.308842 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:56.308844 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:56.308892 (+ 48us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:56.310540 (+ 1648us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.311239 (+ 699us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.311242 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.311244 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.311351 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.311819 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.311823 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.311824 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.312035 (+ 211us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:56.312125 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.312841 (+ 716us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.312972 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:05:56.314388 26407 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.299606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37363 (local address 127.30.204.65:38851)
0807 00:05:56.300096 (+ 490us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:56.300144 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:56.301418 (+ 1274us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:56.301990 (+ 572us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:56.301997 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:56.302404 (+ 407us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:56.303094 (+ 690us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.303118 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.304168 (+ 1050us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.304172 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.305425 (+ 1253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.305438 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.305998 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.306002 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.306206 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.306890 (+ 684us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:56.306911 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:56.308612 (+ 1701us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:56.310680 (+ 2068us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:56.310689 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:56.310711 (+ 22us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:56.311135 (+ 424us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:56.311448 (+ 313us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:56.311474 (+ 26us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:56.311476 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:56.311702 (+ 226us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:56.312141 (+ 439us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:56.312148 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:56.312544 (+ 396us) client_negotiation.cc:769] Sending connection context
0807 00:05:56.312769 (+ 225us) client_negotiation.cc:240] Negotiation successful
0807 00:05:56.313179 (+ 410us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":288,"thread_start_us":132,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:05:56.315848 26405 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37363
I20260807 00:05:56.316112 26405 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:56.316660 26405 heartbeater.cc:511] Master 127.30.204.126:37363 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:05:56.318408 26077 ts_manager.cc:194] Registered new tserver with Master: 2fd0c19e3949458fa1416da81e80cb12 (127.30.204.65:40649)
I20260807 00:05:56.319767 26077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:38851
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:05:56.336457 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37363
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:56.455266 26411 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:56.455605 26411 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:56.455657 26411 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:56.459861 26411 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:56.459936 26411 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:56.460004 26411 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:05:56.465803 26411 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26411
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:56.467029 26411 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:56.468313 26411 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:56.478463 26424 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:56.478466 26421 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:56.478580 26411 server_base.cc:1034] running on GCE node
W20260807 00:05:56.478466 26418 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:56.479286 26411 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:56.480036 26411 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:56.481194 26411 hybrid_clock.cc:648] HybridClock initialized: now 1786061156481166 us; error 46 us; skew 500 ppm
Aug 07 00:05:56 dist-test-slave-pww0 krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061156, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:56.484172 26411 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:05:56.485160 26411 webserver.cc:460] Webserver started at http://127.30.204.66:40207/ using document root <none> and password file <none>
I20260807 00:05:56.485756 26411 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:56.485817 26411 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:56.485987 26411 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:56.488204 26411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "f1fcfaed5db24e75b9be384a66357ca9"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "bd5e507e1820d64272a28bd15d6654ed"
server_key_iv: "0ca93e7232cf729027884083bd8ff5d1"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.488790 26411 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "f1fcfaed5db24e75b9be384a66357ca9"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "bd5e507e1820d64272a28bd15d6654ed"
server_key_iv: "0ca93e7232cf729027884083bd8ff5d1"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.492846 26411 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.000s sys 0.004s
I20260807 00:05:56.495560 26434 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.496984 26411 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:05:56.497115 26411 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "f1fcfaed5db24e75b9be384a66357ca9"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "bd5e507e1820d64272a28bd15d6654ed"
server_key_iv: "0ca93e7232cf729027884083bd8ff5d1"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.497228 26411 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:56.513141 26411 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:56.516414 26411 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:56.516608 26411 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:56.517246 26411 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:56.518154 26411 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:56.518203 26411 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.518261 26411 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:56.518287 26411 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.527540 26411 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:46197
I20260807 00:05:56.527652 26611 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:46197 every 8 connection(s)
I20260807 00:05:56.528554 26411 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:05:56.534680 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 26411
I20260807 00:05:56.534797 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:05:56.535068 31537 external_mini_cluster.cc:1428] Setting key 97747a54320afc685888a1fb774c7ec7
Aug 07 00:05:56 dist-test-slave-pww0 krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061156, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:56.544049 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.530538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:59451 (local address 127.30.204.126:37363)
0807 00:05:56.530691 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:56.530695 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:56.531671 (+ 976us) server_negotiation.cc:407] Connection header received
0807 00:05:56.533059 (+ 1388us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:56.533063 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:56.533124 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:56.533198 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:56.534463 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.535066 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.536595 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.536846 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.536926 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.540034 (+ 3108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:56.540038 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:56.540041 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:56.540075 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:56.541566 (+ 1491us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.542154 (+ 588us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.542158 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.542160 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.542268 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.542689 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.542693 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.542694 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.542972 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:56.543076 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.543696 (+ 620us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.543867 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:05:56.544922 26615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.530968 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37363 (local address 127.30.204.66:59451)
0807 00:05:56.531515 (+ 547us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:56.531573 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:56.532834 (+ 1261us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:56.533407 (+ 573us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:56.533415 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:56.533792 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:56.534310 (+ 518us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.534325 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.535219 (+ 894us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.535223 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.536481 (+ 1258us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.536493 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.537008 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.537012 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.537230 (+ 218us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.537872 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:56.537893 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:56.539830 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:56.541696 (+ 1866us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:56.541703 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:56.541720 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:56.542001 (+ 281us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:56.542395 (+ 394us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:56.542398 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:56.542400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:56.542576 (+ 176us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:56.543090 (+ 514us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:56.543097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:56.543389 (+ 292us) client_negotiation.cc:769] Sending connection context
0807 00:05:56.543599 (+ 210us) client_negotiation.cc:240] Negotiation successful
0807 00:05:56.543951 (+ 352us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":298,"thread_start_us":124,"threads_started":1}
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:05:56.546159 26613 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37363
I20260807 00:05:56.546420 26613 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:56.546963 26613 heartbeater.cc:511] Master 127.30.204.126:37363 requested a full tablet report, sending...
I20260807 00:05:56.548141 26077 ts_manager.cc:194] Registered new tserver with Master: f1fcfaed5db24e75b9be384a66357ca9 (127.30.204.66:46197)
I20260807 00:05:56.548755 26077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:59451
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:05:56.567198 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37363
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:44167
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:56.688536 26621 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:56.688840 26621 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:56.688885 26621 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:56.693080 26621 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:56.693167 26621 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:56.693229 26621 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:05:56.698976 26621 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:44167
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37363
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26621
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:56.700268 26621 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:56.701434 26621 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:56.712071 26640 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:56.712062 26632 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:56.712124 26621 server_base.cc:1034] running on GCE node
W20260807 00:05:56.712067 26634 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:56.712894 26621 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:56.713647 26621 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:56.714851 26621 hybrid_clock.cc:648] HybridClock initialized: now 1786061156714827 us; error 33 us; skew 500 ppm
Aug 07 00:05:56 dist-test-slave-pww0 krb5kdc[26012](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061156, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:56.717856 26621 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:05:56.718869 26621 webserver.cc:460] Webserver started at http://127.30.204.67:35231/ using document root <none> and password file <none>
I20260807 00:05:56.719503 26621 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:56.719585 26621 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:56.719822 26621 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:56.722050 26621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "661b7fbd756e4f30a888bb4f29c705eb"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "6ee7b9c488f24e288444043a648be771"
server_key_iv: "7b14acbb2e99b356764d5b2b9339e273"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.722651 26621 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "661b7fbd756e4f30a888bb4f29c705eb"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "6ee7b9c488f24e288444043a648be771"
server_key_iv: "7b14acbb2e99b356764d5b2b9339e273"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.726960 26621 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.001s
I20260807 00:05:56.729806 26650 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.731292 26621 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:05:56.731438 26621 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "661b7fbd756e4f30a888bb4f29c705eb"
format_stamp: "Formatted at 2026-08-07 00:05:56 on dist-test-slave-pww0"
server_key: "6ee7b9c488f24e288444043a648be771"
server_key_iv: "7b14acbb2e99b356764d5b2b9339e273"
server_key_version: "encryptionkey@0"
I20260807 00:05:56.731652 26621 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:56.745000 26621 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:56.748461 26621 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:56.748637 26621 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:56.749264 26621 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:56.750164 26621 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:56.750216 26621 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.750259 26621 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:56.750300 26621 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:56.760521 26621 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:40141
I20260807 00:05:56.760550 26832 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:40141 every 8 connection(s)
I20260807 00:05:56.761538 26621 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:05:56.765909 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 26621
I20260807 00:05:56.766037 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:05:56.766333 31537 external_mini_cluster.cc:1428] Setting key 44cd93eea2d86402ae6e2e104ea1cd5b
Aug 07 00:05:56 dist-test-slave-pww0 krb5kdc[26012](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061156, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:56.776646 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.763740 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:33413 (local address 127.30.204.126:37363)
0807 00:05:56.763901 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:56.763905 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:56.764912 (+ 1007us) server_negotiation.cc:407] Connection header received
0807 00:05:56.766439 (+ 1527us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:56.766443 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:56.766509 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:56.766596 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:56.768161 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.768757 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.769971 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.770188 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.770289 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.773029 (+ 2740us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:56.773032 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:56.773034 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:56.773063 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:56.774376 (+ 1313us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.774951 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.774954 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.774955 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.775042 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.775405 (+ 363us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.775408 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.775409 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.775660 (+ 251us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:56.775762 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.776328 (+ 566us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.776472 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260807 00:05:56.777562 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.764185 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37363 (local address 127.30.204.67:33413)
0807 00:05:56.764721 (+ 536us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:56.764782 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:56.766179 (+ 1397us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:56.766803 (+ 624us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:56.766811 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:56.767329 (+ 518us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:56.768020 (+ 691us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.768039 (+ 19us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.768888 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.768891 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.769843 (+ 952us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.769855 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.770357 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.770361 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.770549 (+ 188us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.771176 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:05:56.771195 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:56.772841 (+ 1646us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:56.774510 (+ 1669us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:56.774518 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:56.774538 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:56.774833 (+ 295us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:56.775127 (+ 294us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:05:56.775131 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:05:56.775133 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:05:56.775302 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:05:56.775775 (+ 473us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:56.775783 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:56.776066 (+ 283us) client_negotiation.cc:769] Sending connection context
0807 00:05:56.776250 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 00:05:56.776626 (+ 376us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":141,"threads_started":1}
I20260807 00:05:56.778980 26834 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37363
I20260807 00:05:56.779260 26834 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:56.779964 26834 heartbeater.cc:511] Master 127.30.204.126:37363 requested a full tablet report, sending...
I20260807 00:05:56.781054 26077 ts_manager.cc:194] Registered new tserver with Master: 661b7fbd756e4f30a888bb4f29c705eb (127.30.204.67:40141)
I20260807 00:05:56.781633 26077 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:33413
I20260807 00:05:56.792954 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:05:56.802757 26152 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.795162 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52936 (local address 127.30.204.126:37363)
0807 00:05:56.795289 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:56.795294 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:56.795530 (+ 236us) server_negotiation.cc:407] Connection header received
0807 00:05:56.795652 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:56.795656 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:56.795705 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:56.795782 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:56.796530 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.797065 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.797940 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.798165 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.798285 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.799740 (+ 1455us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:56.799743 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:56.799746 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:56.799777 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:56.800965 (+ 1188us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.801407 (+ 442us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.801411 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.801412 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.801498 (+ 86us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:56.801761 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:56.801765 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:56.801767 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:56.802047 (+ 280us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:56.802175 (+ 128us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.802445 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.802576 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:05:56.806095 26077 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52936:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:05:56.809387 26077 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:05:56.825917 26857 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.820171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:46197 (local address 127.0.0.1:50268)
0807 00:05:56.821032 (+ 861us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:56.821046 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:56.821150 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:56.821750 (+ 600us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:56.821753 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:56.821771 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:56.821853 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.821858 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.823111 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.823115 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.824273 (+ 1158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.824283 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.825502 (+ 1219us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.825506 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.825618 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.825639 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 00:05:56.825704 (+ 65us) client_negotiation.cc:240] Negotiation successful
0807 00:05:56.825770 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":795,"mutex_wait_us":23,"thread_start_us":128,"threads_started":1}
I20260807 00:05:56.825992 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.820625 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50268 (local address 127.30.204.66:46197)
0807 00:05:56.821192 (+ 567us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:56.821204 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:56.821226 (+ 22us) server_negotiation.cc:407] Connection header received
0807 00:05:56.821296 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:56.821300 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:56.821452 (+ 152us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:56.821582 (+ 130us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:56.821964 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.822947 (+ 983us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.824409 (+ 1462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.825368 (+ 959us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.825481 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.825616 (+ 135us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.825752 (+ 136us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.825861 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":466,"thread_start_us":126,"threads_started":1}
I20260807 00:05:56.826243 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.820261 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56032 (local address 127.30.204.67:40141)
0807 00:05:56.820768 (+ 507us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:56.820781 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:56.821120 (+ 339us) server_negotiation.cc:407] Connection header received
0807 00:05:56.821210 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:56.821215 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:56.821418 (+ 203us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:56.821582 (+ 164us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:56.821827 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.822752 (+ 925us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.824209 (+ 1457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.825052 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.825181 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.825360 (+ 179us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.825999 (+ 639us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.826110 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":389,"thread_start_us":92,"threads_started":1}
I20260807 00:05:56.826429 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.819540 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.30.204.65:40649)
0807 00:05:56.819964 (+ 424us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:56.819972 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:56.819999 (+ 27us) server_negotiation.cc:407] Connection header received
0807 00:05:56.820210 (+ 211us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:56.820217 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:56.820455 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:56.820613 (+ 158us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:56.821149 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.822293 (+ 1144us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.824131 (+ 1838us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.825021 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.825149 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.825344 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:56.825825 (+ 481us) server_negotiation.cc:299] Negotiation successful
0807 00:05:56.825933 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":328,"thread_start_us":101,"threads_started":1}
I20260807 00:05:56.825928 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.819722 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40141 (local address 127.0.0.1:56032)
0807 00:05:56.820907 (+ 1185us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:56.820922 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:56.821012 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:56.821575 (+ 563us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:56.821578 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:56.821595 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:56.821709 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.821717 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.822883 (+ 1166us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.822887 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.824066 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.824077 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.825435 (+ 1358us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.825440 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.825548 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.825570 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:05:56.825641 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:05:56.825732 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1126,"mutex_wait_us":89,"thread_start_us":111,"threads_started":1}
I20260807 00:05:56.826759 26853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:56.819336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:40649 (local address 127.0.0.1:38364)
0807 00:05:56.819768 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:56.819834 (+ 66us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:56.819976 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:56.820807 (+ 831us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:56.820814 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:56.820840 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:56.821008 (+ 168us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.821015 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.822425 (+ 1410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.822429 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.823978 (+ 1549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:56.823990 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:56.825170 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:56.825174 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:56.825322 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:56.825368 (+ 46us) client_negotiation.cc:769] Sending connection context
0807 00:05:56.825489 (+ 121us) client_negotiation.cc:240] Negotiation successful
0807 00:05:56.825649 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"mutex_wait_us":8,"thread_start_us":143,"threads_started":1}
I20260807 00:05:56.829413 26734 tablet_service.cc:1467] Processing CreateTablet for tablet dfe0418a8a054f64a045fed214285245 (DEFAULT_TABLE table=test-table [id=829044887d7a424bbe4017193a9abe14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:56.829433 26290 tablet_service.cc:1467] Processing CreateTablet for tablet dfe0418a8a054f64a045fed214285245 (DEFAULT_TABLE table=test-table [id=829044887d7a424bbe4017193a9abe14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:56.829437 26509 tablet_service.cc:1467] Processing CreateTablet for tablet dfe0418a8a054f64a045fed214285245 (DEFAULT_TABLE table=test-table [id=829044887d7a424bbe4017193a9abe14]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:56.830600 26734 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfe0418a8a054f64a045fed214285245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:56.830606 26509 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfe0418a8a054f64a045fed214285245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:56.830606 26290 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet dfe0418a8a054f64a045fed214285245. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:56.838304 26861 tablet_bootstrap.cc:492] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Bootstrap starting.
I20260807 00:05:56.839651 26862 tablet_bootstrap.cc:492] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: Bootstrap starting.
I20260807 00:05:56.839933 26863 tablet_bootstrap.cc:492] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Bootstrap starting.
I20260807 00:05:56.841061 26861 tablet_bootstrap.cc:654] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:56.841868 26862 tablet_bootstrap.cc:654] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:56.842152 26863 tablet_bootstrap.cc:654] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:56.842496 26861 log.cc:824] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:56.842635 26862 log.cc:824] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:56.843443 26863 log.cc:824] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:56.844919 26862 tablet_bootstrap.cc:492] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: No bootstrap required, opened a new log
I20260807 00:05:56.844924 26861 tablet_bootstrap.cc:492] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: No bootstrap required, opened a new log
I20260807 00:05:56.845232 26861 ts_tablet_manager.cc:1397] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:05:56.845232 26862 ts_tablet_manager.cc:1397] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:05:56.845340 26863 tablet_bootstrap.cc:492] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: No bootstrap required, opened a new log
I20260807 00:05:56.845531 26863 ts_tablet_manager.cc:1397] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 00:05:56.849293 26863 raft_consensus.cc:348] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:56.849623 26863 raft_consensus.cc:374] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:56.849685 26863 raft_consensus.cc:729] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f1fcfaed5db24e75b9be384a66357ca9, State: Initialized, Role: FOLLOWER
I20260807 00:05:56.849519 26861 raft_consensus.cc:348] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:56.849954 26861 raft_consensus.cc:374] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:56.850037 26861 raft_consensus.cc:729] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 661b7fbd756e4f30a888bb4f29c705eb, State: Initialized, Role: FOLLOWER
I20260807 00:05:56.850204 26863 consensus_queue.cc:260] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:56.850081 26862 raft_consensus.cc:348] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:56.850371 26862 raft_consensus.cc:374] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:56.850422 26862 raft_consensus.cc:729] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2fd0c19e3949458fa1416da81e80cb12, State: Initialized, Role: FOLLOWER
I20260807 00:05:56.850579 26861 consensus_queue.cc:260] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:56.850896 26862 consensus_queue.cc:260] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:56.851052 26613 heartbeater.cc:503] Master 127.30.204.126:37363 was elected leader, sending a full tablet report...
I20260807 00:05:56.851598 26863 ts_tablet_manager.cc:1428] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Time spent starting tablet: real 0.006s user 0.006s sys 0.001s
I20260807 00:05:56.851693 26861 ts_tablet_manager.cc:1428] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Time spent starting tablet: real 0.006s user 0.004s sys 0.004s
I20260807 00:05:56.852061 26834 heartbeater.cc:503] Master 127.30.204.126:37363 was elected leader, sending a full tablet report...
I20260807 00:05:56.851888 26862 ts_tablet_manager.cc:1428] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: Time spent starting tablet: real 0.007s user 0.008s sys 0.000s
I20260807 00:05:56.852397 26405 heartbeater.cc:503] Master 127.30.204.126:37363 was elected leader, sending a full tablet report...
W20260807 00:05:57.012624 26835 tablet.cc:2369] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:05:57.029862 26614 tablet.cc:2369] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:05:57.048677 26406 tablet.cc:2369] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:05:57.062903 26868 raft_consensus.cc:482] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:57.063148 26868 raft_consensus.cc:504] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:57.064426 26868 leader_election.cc:288] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2fd0c19e3949458fa1416da81e80cb12 (127.30.204.65:40649), 661b7fbd756e4f30a888bb4f29c705eb (127.30.204.67:40141)
I20260807 00:05:57.068358 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.064783 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:38701 (local address 127.30.204.67:40141)
0807 00:05:57.064919 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:57.064924 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:57.065080 (+ 156us) server_negotiation.cc:407] Connection header received
0807 00:05:57.065231 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:57.065234 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:57.065283 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:57.065349 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:57.065657 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.066274 (+ 617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.067344 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.067791 (+ 447us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.067911 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.067969 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:57.068149 (+ 180us) server_negotiation.cc:299] Negotiation successful
0807 00:05:57.068227 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:05:57.068377 26872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.064681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:40141 (local address 127.30.204.66:38701)
0807 00:05:57.064986 (+ 305us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:57.065000 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:57.065121 (+ 121us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:57.065368 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:57.065371 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:57.065389 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:57.065531 (+ 142us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.065536 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.066378 (+ 842us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.066382 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.067213 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.067223 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.067911 (+ 688us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.067915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.068026 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.068052 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 00:05:57.068119 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:05:57.068207 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":231,"thread_start_us":91,"threads_started":1}
I20260807 00:05:57.068462 26854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.064813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:51399 (local address 127.30.204.65:40649)
0807 00:05:57.064958 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:57.064962 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:57.065343 (+ 381us) server_negotiation.cc:407] Connection header received
0807 00:05:57.065482 (+ 139us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:57.065486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:57.065532 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:57.065602 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:57.065862 (+ 260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.066428 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.067436 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.067930 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.068043 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.068096 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:57.068264 (+ 168us) server_negotiation.cc:299] Negotiation successful
0807 00:05:57.068336 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:05:57.068504 26874 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.064681 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:40649 (local address 127.30.204.66:51399)
0807 00:05:57.065243 (+ 562us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:57.065255 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:57.065364 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:57.065620 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:57.065622 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:57.065638 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:57.065762 (+ 124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.065768 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.066534 (+ 766us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.066538 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.067317 (+ 779us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.067326 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.068047 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.068058 (+ 11us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.068156 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.068175 (+ 19us) client_negotiation.cc:769] Sending connection context
0807 00:05:57.068239 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 00:05:57.068313 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":501,"mutex_wait_us":300,"thread_start_us":57,"threads_started":1}
I20260807 00:05:57.069295 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe0418a8a054f64a045fed214285245" candidate_uuid: "f1fcfaed5db24e75b9be384a66357ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661b7fbd756e4f30a888bb4f29c705eb" is_pre_election: true
I20260807 00:05:57.069283 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe0418a8a054f64a045fed214285245" candidate_uuid: "f1fcfaed5db24e75b9be384a66357ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd0c19e3949458fa1416da81e80cb12" is_pre_election: true
I20260807 00:05:57.069646 26327 raft_consensus.cc:2454] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1fcfaed5db24e75b9be384a66357ca9 in term 0.
I20260807 00:05:57.069646 26765 raft_consensus.cc:2454] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f1fcfaed5db24e75b9be384a66357ca9 in term 0.
I20260807 00:05:57.070194 26436 leader_election.cc:302] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661b7fbd756e4f30a888bb4f29c705eb, f1fcfaed5db24e75b9be384a66357ca9; no voters:
I20260807 00:05:57.070518 26868 raft_consensus.cc:2781] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:05:57.070613 26868 raft_consensus.cc:482] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:05:57.070781 26868 raft_consensus.cc:3037] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:57.072045 26868 raft_consensus.cc:504] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:57.072486 26868 leader_election.cc:288] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [CANDIDATE]: Term 1 election: Requested vote from peers 2fd0c19e3949458fa1416da81e80cb12 (127.30.204.65:40649), 661b7fbd756e4f30a888bb4f29c705eb (127.30.204.67:40141)
I20260807 00:05:57.073004 26765 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe0418a8a054f64a045fed214285245" candidate_uuid: "f1fcfaed5db24e75b9be384a66357ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "661b7fbd756e4f30a888bb4f29c705eb"
I20260807 00:05:57.073096 26327 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "dfe0418a8a054f64a045fed214285245" candidate_uuid: "f1fcfaed5db24e75b9be384a66357ca9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2fd0c19e3949458fa1416da81e80cb12"
I20260807 00:05:57.073140 26765 raft_consensus.cc:3037] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:57.073257 26327 raft_consensus.cc:3037] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:57.074234 26765 raft_consensus.cc:2454] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1fcfaed5db24e75b9be384a66357ca9 in term 1.
I20260807 00:05:57.074416 26327 raft_consensus.cc:2454] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f1fcfaed5db24e75b9be384a66357ca9 in term 1.
I20260807 00:05:57.074604 26436 leader_election.cc:302] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 661b7fbd756e4f30a888bb4f29c705eb, f1fcfaed5db24e75b9be384a66357ca9; no voters:
I20260807 00:05:57.074795 26868 raft_consensus.cc:2781] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:05:57.075021 26868 raft_consensus.cc:686] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 1 LEADER]: Becoming Leader. State: Replica: f1fcfaed5db24e75b9be384a66357ca9, State: Running, Role: LEADER
I20260807 00:05:57.075392 26868 consensus_queue.cc:237] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } }
I20260807 00:05:57.078799 26075 catalog_manager.cc:5495] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 reported cstate change: term changed from 0 to 1, leader changed from <none> to f1fcfaed5db24e75b9be384a66357ca9 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "f1fcfaed5db24e75b9be384a66357ca9" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f1fcfaed5db24e75b9be384a66357ca9" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 46197 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 } health_report { overall_health: UNKNOWN } } }
I20260807 00:05:57.130312 26859 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.125371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50270 (local address 127.30.204.66:46197)
0807 00:05:57.125498 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:57.125502 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:57.125605 (+ 103us) server_negotiation.cc:407] Connection header received
0807 00:05:57.125847 (+ 242us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:57.125851 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:57.125929 (+ 78us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:57.126053 (+ 124us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:05:57.126316 (+ 263us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.127085 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.128332 (+ 1247us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.128642 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.128752 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.129192 (+ 440us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:05:57.129831 (+ 639us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:05:57.129973 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:57.130050 (+ 77us) server_negotiation.cc:299] Negotiation successful
0807 00:05:57.130135 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 00:05:57.137730 26327 raft_consensus.cc:1261] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 1 FOLLOWER]: Refusing update from remote peer f1fcfaed5db24e75b9be384a66357ca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:05:57.138687 26765 raft_consensus.cc:1261] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 1 FOLLOWER]: Refusing update from remote peer f1fcfaed5db24e75b9be384a66357ca9: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:05:57.138594 26868 consensus_queue.cc:1035] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2fd0c19e3949458fa1416da81e80cb12" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 40649 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:05:57.139259 26868 consensus_queue.cc:1035] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [LEADER]: Connected to new peer: Peer: permanent_uuid: "661b7fbd756e4f30a888bb4f29c705eb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 40141 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:05:57.147629 26880 mvcc.cc:205] Tried to move back new op lower bound from 7315706499627302912 to 7315706499383820288. Current Snapshot: MvccSnapshot[applied={T|T < 7315706499627302912}]
I20260807 00:05:57.150570 26879 mvcc.cc:205] Tried to move back new op lower bound from 7315706499627302912 to 7315706499383820288. Current Snapshot: MvccSnapshot[applied={T|T < 7315706499627302912 or (T in {7315706499627302912})}]
I20260807 00:05:57.150688 26881 mvcc.cc:205] Tried to move back new op lower bound from 7315706499627302912 to 7315706499383820288. Current Snapshot: MvccSnapshot[applied={T|T < 7315706499627302912}]
I20260807 00:05:57.155977 26076 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:05:57.156157 26076 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52936:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:05:57.158931 26076 catalog_manager.cc:5782] T 00000000000000000000000000000000 P afcd0bfc4b8d4076b595c8b2994dc378: Sending DeleteTablet for 3 replicas of tablet dfe0418a8a054f64a045fed214285245
I20260807 00:05:57.159976 26734 tablet_service.cc:1514] Processing DeleteTablet for tablet dfe0418a8a054f64a045fed214285245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:57 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:56032
I20260807 00:05:57.159967 26290 tablet_service.cc:1514] Processing DeleteTablet for tablet dfe0418a8a054f64a045fed214285245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:57 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:38364
I20260807 00:05:57.160601 26895 tablet_replica.cc:330] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12: stopping tablet replica
I20260807 00:05:57.160570 26509 tablet_service.cc:1514] Processing DeleteTablet for tablet dfe0418a8a054f64a045fed214285245 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:57 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:50268
I20260807 00:05:57.161026 26895 raft_consensus.cc:2229] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:05:57.161063 26897 tablet_replica.cc:330] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: stopping tablet replica
I20260807 00:05:57.161478 26895 raft_consensus.cc:2258] T dfe0418a8a054f64a045fed214285245 P 2fd0c19e3949458fa1416da81e80cb12 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:57.161505 26897 raft_consensus.cc:2229] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:05:57.162004 26897 raft_consensus.cc:2258] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:57.162070 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 26186
I20260807 00:05:57.165639 26892 tablet_replica.cc:330] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: stopping tablet replica
I20260807 00:05:57.166018 26892 raft_consensus.cc:2229] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:05:57.166194 26897 ts_tablet_manager.cc:1905] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:05:57.166384 26892 raft_consensus.cc:2258] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:57.169477 26897 ts_tablet_manager.cc:1918] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:05:57.169585 26897 log.cc:1196] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/dfe0418a8a054f64a045fed214285245
I20260807 00:05:57.169876 26897 ts_tablet_manager.cc:1939] T dfe0418a8a054f64a045fed214285245 P f1fcfaed5db24e75b9be384a66357ca9: Deleting consensus metadata
I20260807 00:05:57.171938 26054 catalog_manager.cc:4841] TS f1fcfaed5db24e75b9be384a66357ca9 (127.30.204.66:46197): tablet dfe0418a8a054f64a045fed214285245 (table test-table [id=829044887d7a424bbe4017193a9abe14]) successfully deleted
I20260807 00:05:57.177199 26892 ts_tablet_manager.cc:1905] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260807 00:05:57.179437 26054 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:40649 (error 108)
I20260807 00:05:57.180162 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 26411
I20260807 00:05:57.180388 26892 ts_tablet_manager.cc:1918] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:05:57.180500 26892 log.cc:1196] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/dfe0418a8a054f64a045fed214285245
I20260807 00:05:57.183001 26892 ts_tablet_manager.cc:1939] T dfe0418a8a054f64a045fed214285245 P 661b7fbd756e4f30a888bb4f29c705eb: Deleting consensus metadata
I20260807 00:05:57.184427 26053 catalog_manager.cc:4841] TS 661b7fbd756e4f30a888bb4f29c705eb (127.30.204.67:40141): tablet dfe0418a8a054f64a045fed214285245 (table test-table [id=829044887d7a424bbe4017193a9abe14]) successfully deleted
I20260807 00:05:57.195040 26856 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.193791 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:40649 (local address 127.0.0.1:38370)
0807 00:05:57.194789 (+ 998us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:57.194916 (+ 127us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:40649: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":863}
W20260807 00:05:57.196676 26054 catalog_manager.cc:4584] TS 2fd0c19e3949458fa1416da81e80cb12 (127.30.204.65:40649): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet dfe0418a8a054f64a045fed214285245: Network error: Client connection negotiation failed: client connection to 127.30.204.65:40649: connect: Connection refused (error 111)
I20260807 00:05:57.202028 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 26621
I20260807 00:05:57.214533 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 26030
2026-08-07T00:05:57Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2738 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-08-07T00:05:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:05:57Z Disabled control of system clock
I20260807 00:05:57.256958 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43059
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:43059
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:05:57.377206 26909 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:57.377519 26909 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:57.377557 26909 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:57.381775 26909 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:05:57.381863 26909 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:57.381877 26909 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:57.381896 26909 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:05:57.381910 26909 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:05:57.387095 26909 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:43059
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:43059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26909
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:57.388355 26909 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:57.389845 26909 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:57.399395 26918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:57.399395 26924 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:57.399402 26921 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:57.399534 26909 server_base.cc:1034] running on GCE node
I20260807 00:05:57.400256 26909 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:57.401437 26909 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:57.402588 26909 hybrid_clock.cc:648] HybridClock initialized: now 1786061157402571 us; error 31 us; skew 500 ppm
I20260807 00:05:57.404569 26909 webserver.cc:460] Webserver started at http://127.30.204.126:39033/ using document root <none> and password file <none>
I20260807 00:05:57.405160 26909 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:57.405241 26909 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:57.405462 26909 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:57.407758 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "e389bd28811d46338cb1907b381e4006"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "c1e4f92c88f0c6f3ad7bc5908af65739"
server_key_iv: "e0fd1b0c6fae897eb28929c397da1c20"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.408358 26909 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "e389bd28811d46338cb1907b381e4006"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "c1e4f92c88f0c6f3ad7bc5908af65739"
server_key_iv: "e0fd1b0c6fae897eb28929c397da1c20"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.412465 26909 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:05:57.415199 26936 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:57.416579 26909 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:57.416714 26909 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "e389bd28811d46338cb1907b381e4006"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "c1e4f92c88f0c6f3ad7bc5908af65739"
server_key_iv: "e0fd1b0c6fae897eb28929c397da1c20"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.416834 26909 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:57.428647 26909 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:57.429438 26909 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:57.429625 26909 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:57.436892 26909 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:43059
I20260807 00:05:57.436918 27005 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:43059 every 8 connection(s)
I20260807 00:05:57.437930 26909 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:05:57.441567 27008 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:57.444541 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 26909
I20260807 00:05:57.444686 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:05:57.444962 31537 external_mini_cluster.cc:1428] Setting key ebced306a2daecd98751efbaa0dc7d13
I20260807 00:05:57.448827 27008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006: Bootstrap starting.
I20260807 00:05:57.452066 27008 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:57.453066 27008 log.cc:824] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:57.455621 27008 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006: No bootstrap required, opened a new log
I20260807 00:05:57.459860 27008 raft_consensus.cc:348] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e389bd28811d46338cb1907b381e4006" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43059 } }
I20260807 00:05:57.460192 27008 raft_consensus.cc:374] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:57.460271 27008 raft_consensus.cc:729] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e389bd28811d46338cb1907b381e4006, State: Initialized, Role: FOLLOWER
I20260807 00:05:57.460726 27011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.446640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47102 (local address 127.30.204.126:43059)
0807 00:05:57.447516 (+ 876us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:57.447540 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:57.447594 (+ 54us) server_negotiation.cc:407] Connection header received
0807 00:05:57.448497 (+ 903us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:57.448523 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:57.449090 (+ 567us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:57.449511 (+ 421us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:57.450011 (+ 500us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.451607 (+ 1596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.452633 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.453384 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.453585 (+ 201us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.458134 (+ 4549us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:57.458143 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:57.458166 (+ 23us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:05:57.458794 (+ 628us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:57.458889 (+ 95us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:05:57.458924 (+ 35us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:57.459041 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:57.459232 (+ 191us) server_negotiation.cc:299] Negotiation successful
0807 00:05:57.459604 (+ 372us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":118,"threads_started":1}
I20260807 00:05:57.460786 27008 consensus_queue.cc:260] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e389bd28811d46338cb1907b381e4006" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43059 } }
I20260807 00:05:57.460938 27008 raft_consensus.cc:388] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:05:57.461009 27008 raft_consensus.cc:482] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:05:57.461104 27008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:57.462090 27008 raft_consensus.cc:504] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e389bd28811d46338cb1907b381e4006" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43059 } }
I20260807 00:05:57.462453 27008 leader_election.cc:302] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e389bd28811d46338cb1907b381e4006; no voters:
I20260807 00:05:57.462803 27008 leader_election.cc:288] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:05:57.462916 27015 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:05:57.463127 27015 raft_consensus.cc:686] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [term 1 LEADER]: Becoming Leader. State: Replica: e389bd28811d46338cb1907b381e4006, State: Running, Role: LEADER
I20260807 00:05:57.463435 27015 consensus_queue.cc:237] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e389bd28811d46338cb1907b381e4006" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43059 } }
I20260807 00:05:57.464107 27008 sys_catalog.cc:564] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:05:57.465353 27016 sys_catalog.cc:455] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e389bd28811d46338cb1907b381e4006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e389bd28811d46338cb1907b381e4006" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43059 } } }
I20260807 00:05:57.465363 27017 sys_catalog.cc:455] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e389bd28811d46338cb1907b381e4006. Latest consensus state: current_term: 1 leader_uuid: "e389bd28811d46338cb1907b381e4006" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e389bd28811d46338cb1907b381e4006" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 43059 } } }
I20260807 00:05:57.465484 27016 sys_catalog.cc:458] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:57.465498 27017 sys_catalog.cc:458] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:57.466857 27028 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:05:57.469738 27028 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:05:57.475622 27028 catalog_manager.cc:1347] Generated new cluster ID: 668a6a3497dc4ab590a3d1619e8550e2
I20260807 00:05:57.475733 27028 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:05:57.510177 27028 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:05:57.511219 27028 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:05:57.518824 27028 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006: Generated new TSK 0
I20260807 00:05:57.519634 27028 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:05:57.530012 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43059
--builtin_ntp_servers=127.30.204.84:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:05:57.651450 27046 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:57.651813 27046 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:57.651854 27046 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:57.656067 27046 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:57.656143 27046 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:57.656201 27046 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:05:57.661285 27046 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27046
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:57.662519 27046 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:57.664067 27046 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:57.674593 27062 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:57.674593 27059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:57.674604 27056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:57.674914 27046 server_base.cc:1034] running on GCE node
I20260807 00:05:57.675374 27046 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:57.676111 27046 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:57.677310 27046 hybrid_clock.cc:648] HybridClock initialized: now 1786061157677280 us; error 39 us; skew 500 ppm
I20260807 00:05:57.679564 27046 webserver.cc:460] Webserver started at http://127.30.204.65:40599/ using document root <none> and password file <none>
I20260807 00:05:57.680214 27046 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:57.680315 27046 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:57.680541 27046 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:57.683022 27046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "09bf6b662a37433ea9e70af02c1defdc"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "308bf129f9bfed3b9921641540942cb1"
server_key_iv: "9acfecc38830cc3ecf5712a11b1f6351"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.683679 27046 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "09bf6b662a37433ea9e70af02c1defdc"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "308bf129f9bfed3b9921641540942cb1"
server_key_iv: "9acfecc38830cc3ecf5712a11b1f6351"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.687711 27046 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:05:57.690338 27074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:57.691742 27046 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:05:57.691869 27046 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "09bf6b662a37433ea9e70af02c1defdc"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "308bf129f9bfed3b9921641540942cb1"
server_key_iv: "9acfecc38830cc3ecf5712a11b1f6351"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.691982 27046 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:57.719540 27046 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:57.720366 27046 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:57.720552 27046 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:57.721228 27046 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:57.722292 27046 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:57.722344 27046 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:57.722402 27046 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:57.722432 27046 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:57.731406 27046 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:46349
I20260807 00:05:57.731436 27262 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:46349 every 8 connection(s)
I20260807 00:05:57.732409 27046 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:05:57.738662 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 27046
I20260807 00:05:57.738804 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:05:57.739154 31537 external_mini_cluster.cc:1428] Setting key 1aa1db03d395c711b30b4e3f6abe069b
I20260807 00:05:57.741550 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43059
--builtin_ntp_servers=127.30.204.84:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:05:57.743057 27011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.734349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:45947 (local address 127.30.204.126:43059)
0807 00:05:57.734510 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:57.734514 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:57.735782 (+ 1268us) server_negotiation.cc:407] Connection header received
0807 00:05:57.737271 (+ 1489us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:57.737275 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:57.737340 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:57.737417 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:57.738522 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.739352 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.740851 (+ 1499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.741185 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.741302 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.742569 (+ 1267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:57.742573 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:57.742575 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:05:57.742592 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:57.742618 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:05:57.742628 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:57.742704 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:57.742833 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 00:05:57.742903 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":73}
I20260807 00:05:57.744074 27267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.734772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43059 (local address 127.30.204.65:45947)
0807 00:05:57.735614 (+ 842us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:57.735666 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:57.737017 (+ 1351us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:57.737627 (+ 610us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:57.737634 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:57.737731 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:57.738288 (+ 557us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.738331 (+ 43us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.739541 (+ 1210us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.739545 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.740671 (+ 1126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.740685 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.741384 (+ 699us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.741388 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.741564 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.742258 (+ 694us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:05:57.742260 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:57.742310 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:05:57.742393 (+ 83us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:57.742709 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:57.742715 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:57.742719 (+ 4us) client_negotiation.cc:769] Sending connection context
0807 00:05:57.742817 (+ 98us) client_negotiation.cc:240] Negotiation successful
0807 00:05:57.743087 (+ 270us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"thread_start_us":143,"threads_started":1}
I20260807 00:05:57.745420 27264 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43059
I20260807 00:05:57.745679 27264 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:57.746279 27264 heartbeater.cc:511] Master 127.30.204.126:43059 requested a full tablet report, sending...
I20260807 00:05:57.748025 26960 ts_manager.cc:194] Registered new tserver with Master: 09bf6b662a37433ea9e70af02c1defdc (127.30.204.65:46349)
I20260807 00:05:57.749229 26960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.65:45947
W20260807 00:05:57.860743 27268 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:57.861055 27268 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:57.861123 27268 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:57.865558 27268 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:57.865645 27268 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:57.865731 27268 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:05:57.870910 27268 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27268
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:57.872170 27268 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:57.873678 27268 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:05:57.884334 27268 server_base.cc:1034] running on GCE node
W20260807 00:05:57.884279 27277 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:57.884289 27278 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:57.884274 27280 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:57.885018 27268 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:57.885736 27268 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:57.886911 27268 hybrid_clock.cc:648] HybridClock initialized: now 1786061157886887 us; error 36 us; skew 500 ppm
I20260807 00:05:57.888998 27268 webserver.cc:460] Webserver started at http://127.30.204.66:41933/ using document root <none> and password file <none>
I20260807 00:05:57.889622 27268 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:57.889699 27268 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:57.889901 27268 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:57.892324 27268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "fce4810f478e42ab977e15e3568b5dcf"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "92d692eb0b2b3376f2cab081f6dd73a5"
server_key_iv: "ca3ef48baf472f1c759fcee5cc24cef8"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.892910 27268 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "fce4810f478e42ab977e15e3568b5dcf"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "92d692eb0b2b3376f2cab081f6dd73a5"
server_key_iv: "ca3ef48baf472f1c759fcee5cc24cef8"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.896953 27268 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 00:05:57.899616 27289 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:57.900910 27268 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.001s
I20260807 00:05:57.901039 27268 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "fce4810f478e42ab977e15e3568b5dcf"
format_stamp: "Formatted at 2026-08-07 00:05:57 on dist-test-slave-pww0"
server_key: "92d692eb0b2b3376f2cab081f6dd73a5"
server_key_iv: "ca3ef48baf472f1c759fcee5cc24cef8"
server_key_version: "encryptionkey@0"
I20260807 00:05:57.901160 27268 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:57.922662 27268 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:57.923435 27268 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:57.923673 27268 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:57.924327 27268 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:57.925339 27268 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:57.925388 27268 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:57.925444 27268 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:57.925473 27268 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:57.934391 27268 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:39287
I20260807 00:05:57.934412 27486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:39287 every 8 connection(s)
I20260807 00:05:57.935392 27268 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:05:57.940060 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 27268
I20260807 00:05:57.940189 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:05:57.940474 31537 external_mini_cluster.cc:1428] Setting key b8fcb8c12101195cd8e09aabdcf7598f
I20260807 00:05:57.942795 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43059
--builtin_ntp_servers=127.30.204.84:40777
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:05:57.945814 27011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.937323 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:55299 (local address 127.30.204.126:43059)
0807 00:05:57.937495 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:57.937500 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:57.938713 (+ 1213us) server_negotiation.cc:407] Connection header received
0807 00:05:57.940266 (+ 1553us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:57.940269 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:57.940334 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:57.940412 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:57.941607 (+ 1195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.942288 (+ 681us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.943659 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.943960 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.944081 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.945294 (+ 1213us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:57.945298 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:57.945300 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:05:57.945314 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:57.945337 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:05:57.945345 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:57.945404 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:57.945611 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 00:05:57.945678 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260807 00:05:57.946936 27491 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:57.937784 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43059 (local address 127.30.204.66:55299)
0807 00:05:57.938546 (+ 762us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:57.938591 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:57.940004 (+ 1413us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:57.940634 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:57.940645 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:57.940755 (+ 110us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:57.941431 (+ 676us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.941452 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.942435 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.942438 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.943534 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:57.943546 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:57.944115 (+ 569us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:57.944118 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:57.944280 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:57.944951 (+ 671us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:05:57.944954 (+ 3us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:57.945020 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:05:57.945118 (+ 98us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:57.945420 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:57.945429 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:57.945436 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:05:57.945574 (+ 138us) client_negotiation.cc:240] Negotiation successful
0807 00:05:57.946057 (+ 483us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":334,"thread_start_us":135,"threads_started":1}
I20260807 00:05:57.948179 27487 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43059
I20260807 00:05:57.948483 27487 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:57.949069 27487 heartbeater.cc:511] Master 127.30.204.126:43059 requested a full tablet report, sending...
I20260807 00:05:57.950155 26960 ts_manager.cc:194] Registered new tserver with Master: fce4810f478e42ab977e15e3568b5dcf (127.30.204.66:39287)
I20260807 00:05:57.950766 26960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.66:55299
W20260807 00:05:58.064782 27492 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:58.065111 27492 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:58.065171 27492 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:58.069564 27492 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:58.069655 27492 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:58.069741 27492 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:05:58.075019 27492 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:40777
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:43059
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27492
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:58.076354 27492 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:58.077982 27492 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:58.088846 27502 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:58.088846 27506 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:58.088848 27501 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:58.088907 27492 server_base.cc:1034] running on GCE node
I20260807 00:05:58.089636 27492 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:58.090497 27492 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:58.091699 27492 hybrid_clock.cc:648] HybridClock initialized: now 1786061158091665 us; error 55 us; skew 500 ppm
I20260807 00:05:58.093839 27492 webserver.cc:460] Webserver started at http://127.30.204.67:33953/ using document root <none> and password file <none>
I20260807 00:05:58.094473 27492 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:58.094559 27492 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:58.094794 27492 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:58.097179 27492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "00ff4fe7df7a4059beaac422f4946c63"
format_stamp: "Formatted at 2026-08-07 00:05:58 on dist-test-slave-pww0"
server_key: "c32d066a68447eb925bb17d9fd5a80c1"
server_key_iv: "49fa53e2cec03f921ba4ba49dfa20994"
server_key_version: "encryptionkey@0"
I20260807 00:05:58.097828 27492 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "00ff4fe7df7a4059beaac422f4946c63"
format_stamp: "Formatted at 2026-08-07 00:05:58 on dist-test-slave-pww0"
server_key: "c32d066a68447eb925bb17d9fd5a80c1"
server_key_iv: "49fa53e2cec03f921ba4ba49dfa20994"
server_key_version: "encryptionkey@0"
I20260807 00:05:58.102032 27492 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:05:58.104941 27518 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:58.106578 27492 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:05:58.106726 27492 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "00ff4fe7df7a4059beaac422f4946c63"
format_stamp: "Formatted at 2026-08-07 00:05:58 on dist-test-slave-pww0"
server_key: "c32d066a68447eb925bb17d9fd5a80c1"
server_key_iv: "49fa53e2cec03f921ba4ba49dfa20994"
server_key_version: "encryptionkey@0"
I20260807 00:05:58.106855 27492 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:58.118850 27492 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:58.119683 27492 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:58.119887 27492 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:58.120596 27492 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:05:58.121675 27492 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:05:58.121726 27492 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:58.121786 27492 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:05:58.121815 27492 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:58.130590 27492 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:45853
I20260807 00:05:58.130617 27698 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:45853 every 8 connection(s)
I20260807 00:05:58.131600 27492 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:05:58.132474 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 27492
I20260807 00:05:58.132591 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:05:58.132862 31537 external_mini_cluster.cc:1428] Setting key e9072c40426e54930f913df3d770aaeb
I20260807 00:05:58.141973 27011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.133813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:50715 (local address 127.30.204.126:43059)
0807 00:05:58.133963 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.133968 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.135387 (+ 1419us) server_negotiation.cc:407] Connection header received
0807 00:05:58.136894 (+ 1507us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.136896 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.136949 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.137022 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:58.137998 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.138552 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.139834 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.140083 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.140190 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.141395 (+ 1205us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:58.141399 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:58.141403 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:05:58.141420 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:58.141443 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:05:58.141452 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:58.141517 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.141766 (+ 249us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.141838 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:05:58.142930 27702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.134279 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:43059 (local address 127.30.204.67:50715)
0807 00:05:58.135246 (+ 967us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:58.135308 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:58.136654 (+ 1346us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:58.137181 (+ 527us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:58.137190 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:58.137295 (+ 105us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:05:58.137839 (+ 544us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.137854 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.138675 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.138678 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.139689 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.139702 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.140249 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.140252 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.140407 (+ 155us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.141068 (+ 661us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:05:58.141069 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:05:58.141119 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:05:58.141210 (+ 91us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:05:58.141570 (+ 360us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:05:58.141582 (+ 12us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:05:58.141588 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:05:58.141719 (+ 131us) client_negotiation.cc:240] Negotiation successful
0807 00:05:58.142019 (+ 300us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":422,"thread_start_us":155,"threads_started":1}
I20260807 00:05:58.144287 27699 heartbeater.cc:347] Connected to a master server at 127.30.204.126:43059
I20260807 00:05:58.144534 27699 heartbeater.cc:464] Registering TS with master...
I20260807 00:05:58.145128 27699 heartbeater.cc:511] Master 127.30.204.126:43059 requested a full tablet report, sending...
I20260807 00:05:58.146337 26960 ts_manager.cc:194] Registered new tserver with Master: 00ff4fe7df7a4059beaac422f4946c63 (127.30.204.67:45853)
I20260807 00:05:58.147158 26960 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.67:50715
I20260807 00:05:58.147732 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:05:58.154421 27011 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.150244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47118 (local address 127.30.204.126:43059)
0807 00:05:58.150451 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.150456 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.150625 (+ 169us) server_negotiation.cc:407] Connection header received
0807 00:05:58.150777 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.150780 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.150827 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.150897 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:58.151207 (+ 310us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.151817 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.152693 (+ 876us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.152926 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.153018 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.153974 (+ 956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:58.153977 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:58.153979 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:05:58.153999 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:58.154018 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:05:58.154027 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:58.154089 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.154187 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.154255 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":98}
I20260807 00:05:58.157922 26960 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47118:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:05:58.161193 26960 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:05:58.176891 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.170889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50526 (local address 127.30.204.66:39287)
0807 00:05:58.171697 (+ 808us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.171722 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.171750 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:05:58.171831 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.171837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.172073 (+ 236us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.172242 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:58.172550 (+ 308us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.173688 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.175261 (+ 1573us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.176113 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.176219 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.176444 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.176617 (+ 173us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.176725 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":689,"thread_start_us":139,"threads_started":1}
I20260807 00:05:58.176887 27717 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.170735 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:39287 (local address 127.0.0.1:50526)
0807 00:05:58.171136 (+ 401us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:58.171196 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:58.171351 (+ 155us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:58.172239 (+ 888us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:58.172245 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:58.172268 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:58.172412 (+ 144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.172420 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.173803 (+ 1383us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.173806 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.175099 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.175110 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.176235 (+ 1125us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.176238 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.176404 (+ 166us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.176456 (+ 52us) client_negotiation.cc:769] Sending connection context
0807 00:05:58.176567 (+ 111us) client_negotiation.cc:240] Negotiation successful
0807 00:05:58.176716 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":207,"mutex_wait_us":74,"thread_start_us":94,"threads_started":1}
I20260807 00:05:58.177367 27718 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.170756 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:46349 (local address 127.0.0.1:47600)
0807 00:05:58.171687 (+ 931us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:58.171701 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:58.171801 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:58.172687 (+ 886us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:58.172690 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:58.172709 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:58.172823 (+ 114us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.172830 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.174090 (+ 1260us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.174093 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.175188 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.175201 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.176322 (+ 1121us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.176326 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.176444 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.176471 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:05:58.176547 (+ 76us) client_negotiation.cc:240] Negotiation successful
0807 00:05:58.176721 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":823,"mutex_wait_us":65,"thread_start_us":94,"threads_started":1}
I20260807 00:05:58.177675 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.171705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47600 (local address 127.30.204.65:46349)
0807 00:05:58.172198 (+ 493us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.172209 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.172235 (+ 26us) server_negotiation.cc:407] Connection header received
0807 00:05:58.172316 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.172321 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.172514 (+ 193us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.172968 (+ 454us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:58.173094 (+ 126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.173962 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.175331 (+ 1369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.176184 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.176818 (+ 634us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.177234 (+ 416us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.177292 (+ 58us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.177395 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":391,"thread_start_us":167,"threads_started":1}
I20260807 00:05:58.177755 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.172086 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:45853 (local address 127.0.0.1:44254)
0807 00:05:58.172501 (+ 415us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:58.172514 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:58.172597 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:58.173106 (+ 509us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:58.173108 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:58.173137 (+ 29us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:58.173225 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.173231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.174449 (+ 1218us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.174452 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.175843 (+ 1391us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.175854 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.177141 (+ 1287us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.177144 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.177244 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.177267 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:05:58.177326 (+ 59us) client_negotiation.cc:240] Negotiation successful
0807 00:05:58.177392 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":353,"mutex_wait_us":201,"thread_start_us":110,"threads_started":1}
I20260807 00:05:58.178022 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.171713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44254 (local address 127.30.204.67:45853)
0807 00:05:58.172110 (+ 397us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.172122 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.172685 (+ 563us) server_negotiation.cc:407] Connection header received
0807 00:05:58.172778 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.172783 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.172962 (+ 179us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.173109 (+ 147us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:58.173346 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.174328 (+ 982us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.175977 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.176744 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.176838 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.177595 (+ 757us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.177648 (+ 53us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.177861 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":285,"thread_start_us":161,"threads_started":1}
I20260807 00:05:58.180511 27156 tablet_service.cc:1467] Processing CreateTablet for tablet 16d670525f994b2f810b533e40a5eee3 (DEFAULT_TABLE table=test-table [id=3c0c730f730d443c81da013ed72b8af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:58.180536 27371 tablet_service.cc:1467] Processing CreateTablet for tablet 16d670525f994b2f810b533e40a5eee3 (DEFAULT_TABLE table=test-table [id=3c0c730f730d443c81da013ed72b8af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:58.180691 27588 tablet_service.cc:1467] Processing CreateTablet for tablet 16d670525f994b2f810b533e40a5eee3 (DEFAULT_TABLE table=test-table [id=3c0c730f730d443c81da013ed72b8af1]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:05:58.181620 27156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d670525f994b2f810b533e40a5eee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:58.181648 27371 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d670525f994b2f810b533e40a5eee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:58.181648 27588 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 16d670525f994b2f810b533e40a5eee3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:58.189069 27724 tablet_bootstrap.cc:492] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: Bootstrap starting.
I20260807 00:05:58.190140 27727 tablet_bootstrap.cc:492] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Bootstrap starting.
I20260807 00:05:58.191526 27724 tablet_bootstrap.cc:654] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:58.191885 27728 tablet_bootstrap.cc:492] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Bootstrap starting.
I20260807 00:05:58.192229 27727 tablet_bootstrap.cc:654] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:58.192422 27724 log.cc:824] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:58.193105 27727 log.cc:824] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:58.194191 27728 tablet_bootstrap.cc:654] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:58.194963 27724 tablet_bootstrap.cc:492] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: No bootstrap required, opened a new log
I20260807 00:05:58.195108 27728 log.cc:824] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Log is configured to *not* fsync() on all Append() calls
I20260807 00:05:58.195180 27724 ts_tablet_manager.cc:1397] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 00:05:58.196297 27727 tablet_bootstrap.cc:492] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: No bootstrap required, opened a new log
I20260807 00:05:58.196524 27727 ts_tablet_manager.cc:1397] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 00:05:58.197703 27728 tablet_bootstrap.cc:492] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: No bootstrap required, opened a new log
I20260807 00:05:58.197906 27728 ts_tablet_manager.cc:1397] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 00:05:58.198972 27724 raft_consensus.cc:348] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.199220 27724 raft_consensus.cc:374] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:58.199285 27724 raft_consensus.cc:729] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 09bf6b662a37433ea9e70af02c1defdc, State: Initialized, Role: FOLLOWER
I20260807 00:05:58.199785 27724 consensus_queue.cc:260] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.200538 27264 heartbeater.cc:503] Master 127.30.204.126:43059 was elected leader, sending a full tablet report...
I20260807 00:05:58.200914 27724 ts_tablet_manager.cc:1428] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:05:58.200990 27727 raft_consensus.cc:348] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.201311 27727 raft_consensus.cc:374] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:58.201367 27727 raft_consensus.cc:729] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00ff4fe7df7a4059beaac422f4946c63, State: Initialized, Role: FOLLOWER
I20260807 00:05:58.201885 27727 consensus_queue.cc:260] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.202224 27728 raft_consensus.cc:348] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.202507 27728 raft_consensus.cc:374] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:58.202560 27728 raft_consensus.cc:729] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fce4810f478e42ab977e15e3568b5dcf, State: Initialized, Role: FOLLOWER
I20260807 00:05:58.202721 27699 heartbeater.cc:503] Master 127.30.204.126:43059 was elected leader, sending a full tablet report...
I20260807 00:05:58.202997 27727 ts_tablet_manager.cc:1428] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Time spent starting tablet: real 0.006s user 0.003s sys 0.004s
I20260807 00:05:58.203037 27728 consensus_queue.cc:260] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.203707 27487 heartbeater.cc:503] Master 127.30.204.126:43059 was elected leader, sending a full tablet report...
I20260807 00:05:58.204037 27728 ts_tablet_manager.cc:1428] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
W20260807 00:05:58.233771 27265 tablet.cc:2369] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:05:58.299399 27736 raft_consensus.cc:482] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:05:58.299691 27736 raft_consensus.cc:504] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.300908 27736 leader_election.cc:288] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fce4810f478e42ab977e15e3568b5dcf (127.30.204.66:39287), 00ff4fe7df7a4059beaac422f4946c63 (127.30.204.67:45853)
I20260807 00:05:58.304858 27739 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.301153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:39287 (local address 127.30.204.65:52247)
0807 00:05:58.301509 (+ 356us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:58.301523 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:58.301660 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:58.301907 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:58.301910 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:58.301926 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:58.302090 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.302096 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.302822 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.302825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.303735 (+ 910us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.303745 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.304402 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.304405 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.304506 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.304532 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 00:05:58.304606 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 00:05:58.304687 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":280,"mutex_wait_us":19,"thread_start_us":100,"threads_started":1}
I20260807 00:05:58.304840 27720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.301314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:52247 (local address 127.30.204.66:39287)
0807 00:05:58.301466 (+ 152us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.301471 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.301626 (+ 155us) server_negotiation.cc:407] Connection header received
0807 00:05:58.301769 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.301771 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.301818 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.301882 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:58.302205 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.302708 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.303865 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.304285 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.304401 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.304458 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.304639 (+ 181us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.304720 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:05:58.304927 27722 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.301399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:39229 (local address 127.30.204.67:45853)
0807 00:05:58.301509 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.301514 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.301758 (+ 244us) server_negotiation.cc:407] Connection header received
0807 00:05:58.301882 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.301885 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.301928 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.301993 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:05:58.302316 (+ 323us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.302818 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.303909 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.304364 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.304477 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.304529 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.304723 (+ 194us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.304796 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:05:58.305348 27740 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.301313 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:45853 (local address 127.30.204.65:39229)
0807 00:05:58.301653 (+ 340us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:58.301665 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:05:58.301779 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:05:58.302014 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:05:58.302018 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:05:58.302034 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:05:58.302211 (+ 177us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.302218 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.302923 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.302926 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.303792 (+ 866us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:05:58.303802 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.304504 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.304508 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:05:58.304610 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.304631 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 00:05:58.304695 (+ 64us) client_negotiation.cc:240] Negotiation successful
0807 00:05:58.304770 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":276,"mutex_wait_us":21,"thread_start_us":116,"threads_started":1}
I20260807 00:05:58.305567 27407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d670525f994b2f810b533e40a5eee3" candidate_uuid: "09bf6b662a37433ea9e70af02c1defdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fce4810f478e42ab977e15e3568b5dcf" is_pre_election: true
I20260807 00:05:58.305929 27407 raft_consensus.cc:2454] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6b662a37433ea9e70af02c1defdc in term 0.
I20260807 00:05:58.305986 27616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d670525f994b2f810b533e40a5eee3" candidate_uuid: "09bf6b662a37433ea9e70af02c1defdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ff4fe7df7a4059beaac422f4946c63" is_pre_election: true
I20260807 00:05:58.306321 27616 raft_consensus.cc:2454] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 09bf6b662a37433ea9e70af02c1defdc in term 0.
I20260807 00:05:58.306564 27076 leader_election.cc:302] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 09bf6b662a37433ea9e70af02c1defdc, fce4810f478e42ab977e15e3568b5dcf; no voters:
I20260807 00:05:58.306900 27736 raft_consensus.cc:2781] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:05:58.306988 27736 raft_consensus.cc:482] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:05:58.307021 27736 raft_consensus.cc:3037] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:58.308161 27736 raft_consensus.cc:504] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.308537 27736 leader_election.cc:288] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [CANDIDATE]: Term 1 election: Requested vote from peers fce4810f478e42ab977e15e3568b5dcf (127.30.204.66:39287), 00ff4fe7df7a4059beaac422f4946c63 (127.30.204.67:45853)
I20260807 00:05:58.309053 27616 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d670525f994b2f810b533e40a5eee3" candidate_uuid: "09bf6b662a37433ea9e70af02c1defdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "00ff4fe7df7a4059beaac422f4946c63"
I20260807 00:05:58.309145 27407 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "16d670525f994b2f810b533e40a5eee3" candidate_uuid: "09bf6b662a37433ea9e70af02c1defdc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fce4810f478e42ab977e15e3568b5dcf"
I20260807 00:05:58.309190 27616 raft_consensus.cc:3037] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:58.309299 27407 raft_consensus.cc:3037] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:58.310365 27616 raft_consensus.cc:2454] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6b662a37433ea9e70af02c1defdc in term 1.
I20260807 00:05:58.310380 27407 raft_consensus.cc:2454] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 09bf6b662a37433ea9e70af02c1defdc in term 1.
I20260807 00:05:58.310756 27080 leader_election.cc:302] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 00ff4fe7df7a4059beaac422f4946c63, 09bf6b662a37433ea9e70af02c1defdc; no voters:
I20260807 00:05:58.310937 27736 raft_consensus.cc:2781] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:05:58.311193 27736 raft_consensus.cc:686] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 1 LEADER]: Becoming Leader. State: Replica: 09bf6b662a37433ea9e70af02c1defdc, State: Running, Role: LEADER
I20260807 00:05:58.311628 27736 consensus_queue.cc:237] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } }
I20260807 00:05:58.315603 26958 catalog_manager.cc:5495] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc reported cstate change: term changed from 0 to 1, leader changed from <none> to 09bf6b662a37433ea9e70af02c1defdc (127.30.204.65). New cstate: current_term: 1 leader_uuid: "09bf6b662a37433ea9e70af02c1defdc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "09bf6b662a37433ea9e70af02c1defdc" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46349 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 } health_report { overall_health: UNKNOWN } } }
I20260807 00:05:58.333992 27721 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.330139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47612 (local address 127.30.204.65:46349)
0807 00:05:58.330296 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:58.330300 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:58.330315 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:05:58.330462 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:58.330464 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:58.330528 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:58.330651 (+ 123us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:05:58.330969 (+ 318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.331557 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.332471 (+ 914us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:58.332706 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:58.332801 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:58.333140 (+ 339us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:05:58.333627 (+ 487us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:05:58.333720 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:58.333775 (+ 55us) server_negotiation.cc:299] Negotiation successful
0807 00:05:58.333836 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:05:58.341086 27407 raft_consensus.cc:1261] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 1 FOLLOWER]: Refusing update from remote peer 09bf6b662a37433ea9e70af02c1defdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:05:58.341087 27616 raft_consensus.cc:1261] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 1 FOLLOWER]: Refusing update from remote peer 09bf6b662a37433ea9e70af02c1defdc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:05:58.341924 27736 consensus_queue.cc:1035] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "fce4810f478e42ab977e15e3568b5dcf" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 39287 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:05:58.342162 27742 consensus_queue.cc:1035] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [LEADER]: Connected to new peer: Peer: permanent_uuid: "00ff4fe7df7a4059beaac422f4946c63" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 45853 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:05:58.350994 27746 mvcc.cc:205] Tried to move back new op lower bound from 7315706504556163072 to 7315706504447922176. Current Snapshot: MvccSnapshot[applied={T|T < 7315706504556163072}]
I20260807 00:05:58.351536 27748 mvcc.cc:205] Tried to move back new op lower bound from 7315706504556163072 to 7315706504447922176. Current Snapshot: MvccSnapshot[applied={T|T < 7315706504556163072}]
I20260807 00:05:58.351908 27747 mvcc.cc:205] Tried to move back new op lower bound from 7315706504556163072 to 7315706504447922176. Current Snapshot: MvccSnapshot[applied={T|T < 7315706504556163072}]
I20260807 00:05:58.362106 26960 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:05:58.362340 26960 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47118:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:05:58.365394 26960 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e389bd28811d46338cb1907b381e4006: Sending DeleteTablet for 3 replicas of tablet 16d670525f994b2f810b533e40a5eee3
I20260807 00:05:58.366384 27371 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d670525f994b2f810b533e40a5eee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:58 UTC) from {username='slave'} at 127.0.0.1:50526
I20260807 00:05:58.366382 27156 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d670525f994b2f810b533e40a5eee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:58 UTC) from {username='slave'} at 127.0.0.1:47600
I20260807 00:05:58.367008 27761 tablet_replica.cc:330] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: stopping tablet replica
I20260807 00:05:58.367009 27760 tablet_replica.cc:330] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc: stopping tablet replica
I20260807 00:05:58.366947 27588 tablet_service.cc:1514] Processing DeleteTablet for tablet 16d670525f994b2f810b533e40a5eee3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:05:58 UTC) from {username='slave'} at 127.0.0.1:44254
I20260807 00:05:58.367393 27760 raft_consensus.cc:2229] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:05:58.367391 27761 raft_consensus.cc:2229] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:05:58.367806 27761 raft_consensus.cc:2258] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:58.367893 27760 raft_consensus.cc:2258] T 16d670525f994b2f810b533e40a5eee3 P 09bf6b662a37433ea9e70af02c1defdc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:58.368630 27762 tablet_replica.cc:330] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: stopping tablet replica
I20260807 00:05:58.368641 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 27046
I20260807 00:05:58.368994 27762 raft_consensus.cc:2229] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:05:58.369378 27762 raft_consensus.cc:2258] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:05:58.370519 27761 ts_tablet_manager.cc:1905] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:05:58.372723 27762 ts_tablet_manager.cc:1905] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:05:58.373775 27761 ts_tablet_manager.cc:1918] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:05:58.373878 27761 log.cc:1196] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/16d670525f994b2f810b533e40a5eee3
I20260807 00:05:58.374233 27761 ts_tablet_manager.cc:1939] T 16d670525f994b2f810b533e40a5eee3 P fce4810f478e42ab977e15e3568b5dcf: Deleting consensus metadata
I20260807 00:05:58.374980 27762 ts_tablet_manager.cc:1918] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:05:58.375061 27762 log.cc:1196] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/16d670525f994b2f810b533e40a5eee3
I20260807 00:05:58.375391 27762 ts_tablet_manager.cc:1939] T 16d670525f994b2f810b533e40a5eee3 P 00ff4fe7df7a4059beaac422f4946c63: Deleting consensus metadata
I20260807 00:05:58.376068 26940 catalog_manager.cc:4841] TS fce4810f478e42ab977e15e3568b5dcf (127.30.204.66:39287): tablet 16d670525f994b2f810b533e40a5eee3 (table test-table [id=3c0c730f730d443c81da013ed72b8af1]) successfully deleted
I20260807 00:05:58.377442 26942 catalog_manager.cc:4841] TS 00ff4fe7df7a4059beaac422f4946c63 (127.30.204.67:45853): tablet 16d670525f994b2f810b533e40a5eee3 (table test-table [id=3c0c730f730d443c81da013ed72b8af1]) successfully deleted
I20260807 00:05:58.382692 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 27268
W20260807 00:05:58.382666 26942 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:46349 (error 108)
I20260807 00:05:58.390105 27723 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:58.389613 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:46349 (local address 127.0.0.1:47620)
0807 00:05:58.389877 (+ 264us) negotiation.cc:107] Waiting for socket to connect
0807 00:05:58.389983 (+ 106us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:46349: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":160}
W20260807 00:05:58.390430 26942 catalog_manager.cc:4584] TS 09bf6b662a37433ea9e70af02c1defdc (127.30.204.65:46349): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 16d670525f994b2f810b533e40a5eee3: Network error: Client connection negotiation failed: client connection to 127.30.204.65:46349: connect: Connection refused (error 111)
I20260807 00:05:58.397002 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 27492
I20260807 00:05:58.437119 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 26909
2026-08-07T00:05:58Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (1221 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:05:58 dist-test-slave-pww0 krb5kdc[27767](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:05:58 dist-test-slave-pww0 krb5kdc[27767](info): set up 2 sockets
Aug 07 00:05:58 dist-test-slave-pww0 krb5kdc[27767](info): commencing operation
krb5kdc: starting...
W20260807 00:05:59.603133 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.135s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:05:59 dist-test-slave-pww0 krb5kdc[27767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:05:59Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:05:59Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:05:59.739418 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37167
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:37167
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:05:59.860479 27785 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:05:59.860810 27785 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:05:59.860872 27785 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:05:59.865099 27785 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:05:59.865193 27785 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:05:59.865216 27785 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:05:59.865242 27785 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:05:59.865293 27785 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:05:59.871299 27785 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:37167
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:37167
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27785
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:05:59.872473 27785 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:05:59.873517 27785 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:05:59.882701 27793 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:05:59.882889 27796 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:59.882897 27785 server_base.cc:1034] running on GCE node
W20260807 00:05:59.882697 27801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:05:59.883824 27785 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:05:59.885124 27785 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:05:59.886308 27785 hybrid_clock.cc:648] HybridClock initialized: now 1786061159886287 us; error 41 us; skew 500 ppm
Aug 07 00:05:59 dist-test-slave-pww0 krb5kdc[27767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061159, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:05:59.889367 27785 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:05:59.890419 27785 webserver.cc:460] Webserver started at http://127.30.204.126:37459/ using document root <none> and password file <none>
I20260807 00:05:59.890977 27785 fs_manager.cc:359] Metadata directory not provided
I20260807 00:05:59.891041 27785 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:05:59.891233 27785 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:05:59.893502 27785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "a2fa5ad97a694cf48184be9ea89de194"
format_stamp: "Formatted at 2026-08-07 00:05:59 on dist-test-slave-pww0"
server_key: "34e0c858eb8cf2713afd418aa872fa33"
server_key_iv: "412369efd3811deee8d4b5e14ec993ab"
server_key_version: "encryptionkey@0"
I20260807 00:05:59.894076 27785 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "a2fa5ad97a694cf48184be9ea89de194"
format_stamp: "Formatted at 2026-08-07 00:05:59 on dist-test-slave-pww0"
server_key: "34e0c858eb8cf2713afd418aa872fa33"
server_key_iv: "412369efd3811deee8d4b5e14ec993ab"
server_key_version: "encryptionkey@0"
I20260807 00:05:59.898115 27785 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:05:59.900899 27812 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:05:59.902335 27785 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:05:59.902464 27785 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "a2fa5ad97a694cf48184be9ea89de194"
format_stamp: "Formatted at 2026-08-07 00:05:59 on dist-test-slave-pww0"
server_key: "34e0c858eb8cf2713afd418aa872fa33"
server_key_iv: "412369efd3811deee8d4b5e14ec993ab"
server_key_version: "encryptionkey@0"
I20260807 00:05:59.902595 27785 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:05:59.912079 27785 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:05:59.915246 27785 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:05:59.915438 27785 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:05:59.922678 27785 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:37167
I20260807 00:05:59.922693 27912 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:37167 every 8 connection(s)
I20260807 00:05:59.923732 27785 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:05:59.927163 27914 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:05:59.927855 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 27785
I20260807 00:05:59.927984 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:05:59.928316 31537 external_mini_cluster.cc:1428] Setting key 1ecae272c1a6d85b10d76ba08258d019
I20260807 00:05:59.933796 27914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194: Bootstrap starting.
I20260807 00:05:59.936285 27914 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194: Neither blocks nor log segments found. Creating new log.
I20260807 00:05:59.937206 27914 log.cc:824] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194: Log is configured to *not* fsync() on all Append() calls
Aug 07 00:05:59 dist-test-slave-pww0 krb5kdc[27767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061159, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:05:59.939704 27914 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194: No bootstrap required, opened a new log
I20260807 00:05:59.943017 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:05:59.929868 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41808 (local address 127.30.204.126:37167)
0807 00:05:59.930358 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 00:05:59.930374 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:05:59.930412 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:05:59.931068 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:05:59.931085 (+ 17us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:05:59.931444 (+ 359us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:05:59.931826 (+ 382us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:05:59.932682 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:59.933668 (+ 986us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:59.934567 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:05:59.935053 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:05:59.935213 (+ 160us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:05:59.938015 (+ 2802us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:05:59.938022 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:05:59.938041 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:05:59.938077 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:05:59.939856 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:59.940442 (+ 586us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:59.940448 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:59.940453 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:59.940578 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:05:59.940899 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:05:59.940902 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:05:59.940903 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:05:59.941362 (+ 459us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:05:59.941519 (+ 157us) server_negotiation.cc:1035] Waiting for connection context
0807 00:05:59.941847 (+ 328us) server_negotiation.cc:299] Negotiation successful
0807 00:05:59.942134 (+ 287us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":310,"thread_start_us":132,"threads_started":1}
I20260807 00:05:59.943928 27914 raft_consensus.cc:348] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2fa5ad97a694cf48184be9ea89de194" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37167 } }
I20260807 00:05:59.944234 27914 raft_consensus.cc:374] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:05:59.944298 27914 raft_consensus.cc:729] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a2fa5ad97a694cf48184be9ea89de194, State: Initialized, Role: FOLLOWER
I20260807 00:05:59.944800 27914 consensus_queue.cc:260] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2fa5ad97a694cf48184be9ea89de194" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37167 } }
I20260807 00:05:59.944964 27914 raft_consensus.cc:388] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:05:59.945029 27914 raft_consensus.cc:482] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:05:59.945140 27914 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:05:59.946167 27914 raft_consensus.cc:504] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2fa5ad97a694cf48184be9ea89de194" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37167 } }
I20260807 00:05:59.946513 27914 leader_election.cc:302] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a2fa5ad97a694cf48184be9ea89de194; no voters:
I20260807 00:05:59.946825 27914 leader_election.cc:288] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:05:59.946955 27920 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:05:59.947165 27920 raft_consensus.cc:686] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [term 1 LEADER]: Becoming Leader. State: Replica: a2fa5ad97a694cf48184be9ea89de194, State: Running, Role: LEADER
I20260807 00:05:59.947528 27920 consensus_queue.cc:237] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2fa5ad97a694cf48184be9ea89de194" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37167 } }
I20260807 00:05:59.948199 27914 sys_catalog.cc:564] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:05:59.949244 27922 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a2fa5ad97a694cf48184be9ea89de194. Latest consensus state: current_term: 1 leader_uuid: "a2fa5ad97a694cf48184be9ea89de194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2fa5ad97a694cf48184be9ea89de194" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37167 } } }
I20260807 00:05:59.949378 27922 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [sys.catalog]: This master's current role is: LEADER
I20260807 00:05:59.949867 27921 sys_catalog.cc:455] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a2fa5ad97a694cf48184be9ea89de194" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a2fa5ad97a694cf48184be9ea89de194" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 37167 } } }
I20260807 00:05:59.949978 27921 sys_catalog.cc:458] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194 [sys.catalog]: This master's current role is: LEADER
W20260807 00:05:59.953226 27947 catalog_manager.cc:1558] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:05:59.953327 27947 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:05:59.953488 27933 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:05:59.954512 27933 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:05:59.960870 27933 catalog_manager.cc:1347] Generated new cluster ID: 48cf487e17f24c2d996b3ee24a4b0a9c
I20260807 00:05:59.960975 27933 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:05:59.985353 27933 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:05:59.986586 27933 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:05:59.993093 27933 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194: Generated new TSK 0
I20260807 00:05:59.993899 27933 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:06:00.034211 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37167
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:00.182401 27955 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:00.182710 27955 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:00.182755 27955 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:00.186893 27955 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:00.186980 27955 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:00.187040 27955 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:06:00.193070 27955 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27955
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:00.194300 27955 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:00.195443 27955 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:00.206060 27963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:00.206069 27962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:00.206177 27966 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:00.206519 27955 server_base.cc:1034] running on GCE node
I20260807 00:06:00.207055 27955 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:00.207835 27955 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:00.209002 27955 hybrid_clock.cc:648] HybridClock initialized: now 1786061160208966 us; error 52 us; skew 500 ppm
Aug 07 00:06:00 dist-test-slave-pww0 krb5kdc[27767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:00.212179 27955 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:06:00.213279 27955 webserver.cc:460] Webserver started at http://127.30.204.65:40633/ using document root <none> and password file <none>
I20260807 00:06:00.213901 27955 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:00.213981 27955 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:00.214224 27955 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:00.216713 27955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "0cbfb4198aa14b7bb5b7083180d8713b"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "182e991ebe122a945dcf08f1d56f424a"
server_key_iv: "33af417ab8ec3ca633a395e6433f6109"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.217469 27955 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "0cbfb4198aa14b7bb5b7083180d8713b"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "182e991ebe122a945dcf08f1d56f424a"
server_key_iv: "33af417ab8ec3ca633a395e6433f6109"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.224911 27955 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.006s sys 0.000s
I20260807 00:06:00.228626 27976 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.230430 27955 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:06:00.230589 27955 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "0cbfb4198aa14b7bb5b7083180d8713b"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "182e991ebe122a945dcf08f1d56f424a"
server_key_iv: "33af417ab8ec3ca633a395e6433f6109"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.230747 27955 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:00.264634 27955 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:00.268985 27955 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:00.269219 27955 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:00.270015 27955 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:00.271160 27955 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:00.271235 27955 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.271293 27955 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:00.271322 27955 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.282636 27955 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:34563
I20260807 00:06:00.282667 28196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:34563 every 8 connection(s)
I20260807 00:06:00.283690 27955 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:06:00.289877 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 27955
I20260807 00:06:00.290021 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:06:00.290383 31537 external_mini_cluster.cc:1428] Setting key 3204b334943800be77e522dbff456860
Aug 07 00:06:00 dist-test-slave-pww0 krb5kdc[27767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:00.299711 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.285660 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:38983 (local address 127.30.204.126:37167)
0807 00:06:00.285818 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.285822 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.286931 (+ 1109us) server_negotiation.cc:407] Connection header received
0807 00:06:00.288333 (+ 1402us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.288336 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.288400 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.288483 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:00.290114 (+ 1631us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.290890 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.292536 (+ 1646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.292917 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.293028 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.295958 (+ 2930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:00.295962 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:00.295964 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:00.295992 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:00.297390 (+ 1398us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.297962 (+ 572us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.297965 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.297966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.298051 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.298451 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.298454 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.298455 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.298661 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:00.298762 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.299333 (+ 571us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.299518 (+ 185us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:06:00.300621 28200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.286174 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37167 (local address 127.30.204.65:38983)
0807 00:06:00.286763 (+ 589us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.286813 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.288093 (+ 1280us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.288688 (+ 595us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.288696 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.289113 (+ 417us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:00.289886 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.289910 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.291040 (+ 1130us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.291044 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.292405 (+ 1361us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.292418 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.293104 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.293108 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.293329 (+ 221us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.294057 (+ 728us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:00.294079 (+ 22us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:00.295758 (+ 1679us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:00.297528 (+ 1770us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:00.297535 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:00.297555 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:00.297844 (+ 289us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:00.298139 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:00.298142 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:00.298144 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:00.298350 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:00.298778 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:00.298787 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:00.299071 (+ 284us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.299255 (+ 184us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.299675 (+ 420us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":355,"thread_start_us":158,"threads_started":1}
I20260807 00:06:00.301927 28198 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37167
I20260807 00:06:00.302229 28198 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:00.302806 28198 heartbeater.cc:511] Master 127.30.204.126:37167 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:06:00.304558 27855 ts_manager.cc:194] Registered new tserver with Master: 0cbfb4198aa14b7bb5b7083180d8713b (127.30.204.65:34563)
I20260807 00:06:00.305786 27855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:38983
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:06:00.327076 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37167
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:00.450313 28204 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:00.450613 28204 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:00.450659 28204 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:00.454921 28204 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:00.455018 28204 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:00.455081 28204 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:06:00.460947 28204 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28204
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:00.462231 28204 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:00.463380 28204 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:00.474030 28213 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:00.474058 28214 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:00.474058 28204 server_base.cc:1034] running on GCE node
W20260807 00:06:00.474069 28216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:00.474814 28204 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:00.475610 28204 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:00.476821 28204 hybrid_clock.cc:648] HybridClock initialized: now 1786061160476794 us; error 37 us; skew 500 ppm
Aug 07 00:06:00 dist-test-slave-pww0 krb5kdc[27767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:00.480173 28204 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:06:00.481256 28204 webserver.cc:460] Webserver started at http://127.30.204.66:33035/ using document root <none> and password file <none>
I20260807 00:06:00.481839 28204 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:00.481904 28204 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:00.482085 28204 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:00.484433 28204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "fddf4d40ba104acbb6e0e37192698e03"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "cd708610470a0e4939f434777e320987"
server_key_iv: "96d01698876b2c3d512707698e5d02bd"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.485065 28204 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "fddf4d40ba104acbb6e0e37192698e03"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "cd708610470a0e4939f434777e320987"
server_key_iv: "96d01698876b2c3d512707698e5d02bd"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.489320 28204 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:06:00.492188 28232 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.493722 28204 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:00.493860 28204 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "fddf4d40ba104acbb6e0e37192698e03"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "cd708610470a0e4939f434777e320987"
server_key_iv: "96d01698876b2c3d512707698e5d02bd"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.493992 28204 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:00.511777 28204 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:00.515057 28204 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:00.515255 28204 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:00.515970 28204 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:00.516907 28204 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:00.516960 28204 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.517019 28204 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:00.517050 28204 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.526330 28204 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:41841
I20260807 00:06:00.526463 28413 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:41841 every 8 connection(s)
I20260807 00:06:00.527374 28204 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:06:00.536870 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 28204
I20260807 00:06:00.537016 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:06:00.537361 31537 external_mini_cluster.cc:1428] Setting key e75aac3a6d20246313de1e5d541823ad
Aug 07 00:06:00 dist-test-slave-pww0 krb5kdc[27767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:00.543740 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.529351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:56197 (local address 127.30.204.126:37167)
0807 00:06:00.529534 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.529538 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.530544 (+ 1006us) server_negotiation.cc:407] Connection header received
0807 00:06:00.532157 (+ 1613us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.532161 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.532235 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.532337 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:00.533839 (+ 1502us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.534429 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.535932 (+ 1503us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.536193 (+ 261us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.536313 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.539646 (+ 3333us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:00.539649 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:00.539651 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:00.539681 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:00.541074 (+ 1393us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.541938 (+ 864us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.541943 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.541945 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.542068 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.542423 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.542427 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.542428 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.542691 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:00.542801 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.543392 (+ 591us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.543573 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 00:06:00.544642 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.529798 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37167 (local address 127.30.204.66:56197)
0807 00:06:00.530357 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.530411 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.531797 (+ 1386us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.532553 (+ 756us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.532562 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.532992 (+ 430us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:00.533613 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.533657 (+ 44us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.534660 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.534664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.535759 (+ 1095us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.535774 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.536437 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.536443 (+ 6us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.536743 (+ 300us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.537497 (+ 754us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:00.537522 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:00.539414 (+ 1892us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:00.541376 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:00.541385 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:00.541410 (+ 25us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:00.541784 (+ 374us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:00.542162 (+ 378us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:00.542164 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:00.542165 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:00.542327 (+ 162us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:00.542808 (+ 481us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:00.542814 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:00.543110 (+ 296us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.543283 (+ 173us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.543647 (+ 364us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":329,"thread_start_us":137,"threads_started":1}
I20260807 00:06:00.546126 28414 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37167
I20260807 00:06:00.546409 28414 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:00.546986 28414 heartbeater.cc:511] Master 127.30.204.126:37167 requested a full tablet report, sending...
I20260807 00:06:00.548250 27855 ts_manager.cc:194] Registered new tserver with Master: fddf4d40ba104acbb6e0e37192698e03 (127.30.204.66:41841)
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:06:00.548893 27855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:56197
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:06:00.568714 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37167
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:36389
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:00.688346 28421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:00.688637 28421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:00.688679 28421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:00.692924 28421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:00.693004 28421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:00.693068 28421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:06:00.698889 28421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:36389
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:37167
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:00.700168 28421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:00.701272 28421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:00.711616 28430 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:00.711598 28426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:00.711658 28428 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:00.712018 28421 server_base.cc:1034] running on GCE node
I20260807 00:06:00.712589 28421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:00.713339 28421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:00.714532 28421 hybrid_clock.cc:648] HybridClock initialized: now 1786061160714491 us; error 54 us; skew 500 ppm
Aug 07 00:06:00 dist-test-slave-pww0 krb5kdc[27767](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:00.717612 28421 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:06:00.718642 28421 webserver.cc:460] Webserver started at http://127.30.204.67:41557/ using document root <none> and password file <none>
I20260807 00:06:00.719233 28421 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:00.719305 28421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:00.719571 28421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:00.721792 28421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "f8522f297d6949c19e5b5d76222e4d94"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "f630982ffaebbd36396965cac5c21962"
server_key_iv: "0db1174b0987dad4d07cda4202f27021"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.722399 28421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "f8522f297d6949c19e5b5d76222e4d94"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "f630982ffaebbd36396965cac5c21962"
server_key_iv: "0db1174b0987dad4d07cda4202f27021"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.726583 28421 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:06:00.729348 28440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.730827 28421 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.000s sys 0.002s
I20260807 00:06:00.730976 28421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "f8522f297d6949c19e5b5d76222e4d94"
format_stamp: "Formatted at 2026-08-07 00:06:00 on dist-test-slave-pww0"
server_key: "f630982ffaebbd36396965cac5c21962"
server_key_iv: "0db1174b0987dad4d07cda4202f27021"
server_key_version: "encryptionkey@0"
I20260807 00:06:00.731092 28421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:00.745424 28421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:00.749063 28421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:00.749257 28421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:00.749946 28421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:00.750888 28421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:00.750942 28421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.751001 28421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:00.751026 28421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:00.760394 28421 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:34715
I20260807 00:06:00.760468 28628 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:34715 every 8 connection(s)
I20260807 00:06:00.761399 28421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:06:00.766983 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 28421
I20260807 00:06:00.767131 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:06:00.767517 31537 external_mini_cluster.cc:1428] Setting key dc1ab205d0c1971c13434fe0efe83348
Aug 07 00:06:00 dist-test-slave-pww0 krb5kdc[27767](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061160, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:00.776894 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.763527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:56027 (local address 127.30.204.126:37167)
0807 00:06:00.763671 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.763676 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.764764 (+ 1088us) server_negotiation.cc:407] Connection header received
0807 00:06:00.766391 (+ 1627us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.766395 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.766464 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.766542 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:00.768346 (+ 1804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.769049 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.770323 (+ 1274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.770555 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.770650 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.773232 (+ 2582us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:00.773236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:00.773238 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:00.773267 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:00.774552 (+ 1285us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.775161 (+ 609us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.775164 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.775165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.775258 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.775673 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.775677 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.775678 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.775883 (+ 205us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:00.775975 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.776553 (+ 578us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.776704 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260807 00:06:00.777806 28632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.764040 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:37167 (local address 127.30.204.67:56027)
0807 00:06:00.764559 (+ 519us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.764609 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.766104 (+ 1495us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.766808 (+ 704us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.766818 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.767350 (+ 532us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:00.768144 (+ 794us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.768168 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.769192 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.769195 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.770192 (+ 997us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.770203 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.770714 (+ 511us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.770719 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.770902 (+ 183us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.771509 (+ 607us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:00.771532 (+ 23us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:00.773043 (+ 1511us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:00.774703 (+ 1660us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:00.774711 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:00.774731 (+ 20us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:00.775036 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:00.775347 (+ 311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:00.775350 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:00.775352 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:00.775568 (+ 216us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:00.775991 (+ 423us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:00.775999 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:00.776293 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.776472 (+ 179us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.776865 (+ 393us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":297,"thread_start_us":115,"threads_started":1}
I20260807 00:06:00.779210 28630 heartbeater.cc:347] Connected to a master server at 127.30.204.126:37167
I20260807 00:06:00.779512 28630 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:00.780118 28630 heartbeater.cc:511] Master 127.30.204.126:37167 requested a full tablet report, sending...
I20260807 00:06:00.781365 27855 ts_manager.cc:194] Registered new tserver with Master: f8522f297d6949c19e5b5d76222e4d94 (127.30.204.67:34715)
I20260807 00:06:00.781908 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:06:00.782052 27855 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:56027
I20260807 00:06:00.791916 27918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.784123 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41814 (local address 127.30.204.126:37167)
0807 00:06:00.784245 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.784250 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.784463 (+ 213us) server_negotiation.cc:407] Connection header received
0807 00:06:00.784578 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.784581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.784626 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.784693 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:00.785444 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.785969 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.786797 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.787023 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.787110 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.788753 (+ 1643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:00.788757 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:00.788760 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:00.788788 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:00.789989 (+ 1201us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.790513 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.790517 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.790519 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.790599 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:00.790925 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:00.790928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:00.790930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:00.791149 (+ 219us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:00.791301 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.791598 (+ 297us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.791697 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260807 00:06:00.795224 27855 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41814:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:06:00.798537 27855 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:06:00.814317 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.807911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60832 (local address 127.30.204.66:41841)
0807 00:06:00.808238 (+ 327us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.808252 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.808280 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:06:00.808561 (+ 281us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.808569 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.808826 (+ 257us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.808999 (+ 173us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:00.809541 (+ 542us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.810706 (+ 1165us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.812271 (+ 1565us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.813175 (+ 904us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.813275 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.813449 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.814016 (+ 567us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.814139 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":223,"thread_start_us":121,"threads_started":1}
I20260807 00:06:00.814320 28649 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.808414 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:34715 (local address 127.0.0.1:60164)
0807 00:06:00.808750 (+ 336us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.808763 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.808873 (+ 110us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.809848 (+ 975us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.809851 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.809872 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:00.809981 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.809987 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.811320 (+ 1333us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.811324 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.812393 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.812404 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.813557 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.813560 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.813766 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.813833 (+ 67us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.813953 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.814122 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":274,"mutex_wait_us":108,"thread_start_us":113,"threads_started":1}
I20260807 00:06:00.814709 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.808805 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52140 (local address 127.30.204.65:34563)
0807 00:06:00.809096 (+ 291us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.809102 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.809820 (+ 718us) server_negotiation.cc:407] Connection header received
0807 00:06:00.809908 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.809913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.810095 (+ 182us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.810248 (+ 153us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:00.810617 (+ 369us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.811722 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.813096 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.813882 (+ 786us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.813979 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.814129 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.814433 (+ 304us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.814547 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":199,"thread_start_us":133,"threads_started":1}
I20260807 00:06:00.814821 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.808933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60164 (local address 127.30.204.67:34715)
0807 00:06:00.809349 (+ 416us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.809369 (+ 20us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.809393 (+ 24us) server_negotiation.cc:407] Connection header received
0807 00:06:00.809469 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.809475 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.809685 (+ 210us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.810126 (+ 441us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:00.810289 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.811194 (+ 905us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.812536 (+ 1342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.813403 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.813547 (+ 144us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.813695 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.814565 (+ 870us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.814676 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":316,"thread_start_us":200,"threads_started":1}
I20260807 00:06:00.814320 28647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.807752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:41841 (local address 127.0.0.1:60832)
0807 00:06:00.808151 (+ 399us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.808207 (+ 56us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.808342 (+ 135us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.809206 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.809212 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.809239 (+ 27us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:00.809401 (+ 162us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.809410 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.810841 (+ 1431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.810844 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.812149 (+ 1305us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.812159 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.813515 (+ 1356us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.813519 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.813762 (+ 243us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.813833 (+ 71us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.813953 (+ 120us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.814120 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":206,"mutex_wait_us":53,"thread_start_us":114,"threads_started":1}
I20260807 00:06:00.815136 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.808606 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:34563 (local address 127.0.0.1:52140)
0807 00:06:00.809608 (+ 1002us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.809622 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.809719 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.810414 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.810417 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.810434 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:00.810501 (+ 67us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.810505 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.811855 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.811858 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.812954 (+ 1096us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.812965 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.814199 (+ 1234us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.814203 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.814289 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.814313 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.814746 (+ 433us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.815009 (+ 263us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":936,"thread_start_us":112,"threads_started":1}
I20260807 00:06:00.818130 28306 tablet_service.cc:1467] Processing CreateTablet for tablet 65c6504da98f491a9d5efdca46ede399 (DEFAULT_TABLE table=test-table [id=60e87e16e74443f3a9366c7dd212701f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:00.818151 28510 tablet_service.cc:1467] Processing CreateTablet for tablet 65c6504da98f491a9d5efdca46ede399 (DEFAULT_TABLE table=test-table [id=60e87e16e74443f3a9366c7dd212701f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:00.818145 28035 tablet_service.cc:1467] Processing CreateTablet for tablet 65c6504da98f491a9d5efdca46ede399 (DEFAULT_TABLE table=test-table [id=60e87e16e74443f3a9366c7dd212701f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:00.819312 28306 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c6504da98f491a9d5efdca46ede399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:00.819314 28510 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c6504da98f491a9d5efdca46ede399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:00.819314 28035 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 65c6504da98f491a9d5efdca46ede399. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:00.827450 28656 tablet_bootstrap.cc:492] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b: Bootstrap starting.
I20260807 00:06:00.827863 28655 tablet_bootstrap.cc:492] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Bootstrap starting.
I20260807 00:06:00.828099 28657 tablet_bootstrap.cc:492] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Bootstrap starting.
I20260807 00:06:00.829890 28655 tablet_bootstrap.cc:654] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:00.829965 28656 tablet_bootstrap.cc:654] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:00.830171 28657 tablet_bootstrap.cc:654] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:00.830956 28656 log.cc:824] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:00.831262 28655 log.cc:824] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:00.831370 28657 log.cc:824] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:00.833382 28656 tablet_bootstrap.cc:492] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b: No bootstrap required, opened a new log
I20260807 00:06:00.833622 28656 ts_tablet_manager.cc:1397] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:06:00.833767 28657 tablet_bootstrap.cc:492] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: No bootstrap required, opened a new log
I20260807 00:06:00.833767 28655 tablet_bootstrap.cc:492] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: No bootstrap required, opened a new log
I20260807 00:06:00.833987 28657 ts_tablet_manager.cc:1397] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 00:06:00.833987 28655 ts_tablet_manager.cc:1397] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:06:00.837718 28656 raft_consensus.cc:348] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.838016 28656 raft_consensus.cc:374] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:00.838074 28656 raft_consensus.cc:729] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0cbfb4198aa14b7bb5b7083180d8713b, State: Initialized, Role: FOLLOWER
I20260807 00:06:00.838104 28657 raft_consensus.cc:348] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.838104 28655 raft_consensus.cc:348] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.838389 28657 raft_consensus.cc:374] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:00.838389 28655 raft_consensus.cc:374] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:00.838447 28655 raft_consensus.cc:729] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fddf4d40ba104acbb6e0e37192698e03, State: Initialized, Role: FOLLOWER
I20260807 00:06:00.838459 28657 raft_consensus.cc:729] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f8522f297d6949c19e5b5d76222e4d94, State: Initialized, Role: FOLLOWER
I20260807 00:06:00.838624 28656 consensus_queue.cc:260] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.838930 28655 consensus_queue.cc:260] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.838930 28657 consensus_queue.cc:260] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.839864 28656 ts_tablet_manager.cc:1428] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b: Time spent starting tablet: real 0.006s user 0.001s sys 0.006s
I20260807 00:06:00.840006 28655 ts_tablet_manager.cc:1428] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:06:00.840006 28657 ts_tablet_manager.cc:1428] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Time spent starting tablet: real 0.006s user 0.006s sys 0.002s
I20260807 00:06:00.840263 28198 heartbeater.cc:503] Master 127.30.204.126:37167 was elected leader, sending a full tablet report...
I20260807 00:06:00.840364 28414 heartbeater.cc:503] Master 127.30.204.126:37167 was elected leader, sending a full tablet report...
I20260807 00:06:00.840364 28630 heartbeater.cc:503] Master 127.30.204.126:37167 was elected leader, sending a full tablet report...
I20260807 00:06:00.978575 28666 raft_consensus.cc:482] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:06:00.978824 28666 raft_consensus.cc:504] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.980098 28666 leader_election.cc:288] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 0cbfb4198aa14b7bb5b7083180d8713b (127.30.204.65:34563), f8522f297d6949c19e5b5d76222e4d94 (127.30.204.67:34715)
I20260807 00:06:00.983951 28417 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.980411 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:34715 (local address 127.30.204.66:38785)
0807 00:06:00.980552 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.980569 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.980677 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.980998 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.981001 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.981024 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:00.981146 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.981152 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.981878 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.981882 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.982762 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.982772 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.983500 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.983503 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.983614 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.983637 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.983704 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.983789 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 00:06:00.983952 28652 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.980531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:38785 (local address 127.30.204.67:34715)
0807 00:06:00.980770 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.980775 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.980790 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:06:00.980842 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.980844 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.980890 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.980970 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:00.981259 (+ 289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.981772 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.982886 (+ 1114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.983357 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.983500 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.983565 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.983768 (+ 203us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.983852 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138}
I20260807 00:06:00.984515 28668 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.980417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:34563 (local address 127.30.204.66:41353)
0807 00:06:00.981141 (+ 724us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:00.981153 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:00.981267 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:00.981489 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:00.981491 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:00.981508 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:00.981633 (+ 125us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.981638 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.982342 (+ 704us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.982346 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.983178 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:00.983188 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.983960 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.983964 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:00.984218 (+ 254us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.984245 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:06:00.984319 (+ 74us) client_negotiation.cc:240] Negotiation successful
0807 00:06:00.984383 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":650,"mutex_wait_us":347,"thread_start_us":122,"threads_started":1}
I20260807 00:06:00.984555 28651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:00.980562 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:41353 (local address 127.30.204.65:34563)
0807 00:06:00.980693 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:00.980698 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:00.981257 (+ 559us) server_negotiation.cc:407] Connection header received
0807 00:06:00.981355 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:00.981358 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:00.981403 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:00.981465 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:00.981753 (+ 288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.982230 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.983314 (+ 1084us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:00.983843 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:00.983960 (+ 117us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:00.984012 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:00.984348 (+ 336us) server_negotiation.cc:299] Negotiation successful
0807 00:06:00.984440 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260807 00:06:00.984867 28545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6504da98f491a9d5efdca46ede399" candidate_uuid: "fddf4d40ba104acbb6e0e37192698e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8522f297d6949c19e5b5d76222e4d94" is_pre_election: true
I20260807 00:06:00.985126 28099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6504da98f491a9d5efdca46ede399" candidate_uuid: "fddf4d40ba104acbb6e0e37192698e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" is_pre_election: true
I20260807 00:06:00.985203 28545 raft_consensus.cc:2454] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fddf4d40ba104acbb6e0e37192698e03 in term 0.
I20260807 00:06:00.985396 28099 raft_consensus.cc:2454] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate fddf4d40ba104acbb6e0e37192698e03 in term 0.
I20260807 00:06:00.985752 28237 leader_election.cc:302] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: f8522f297d6949c19e5b5d76222e4d94, fddf4d40ba104acbb6e0e37192698e03; no voters:
I20260807 00:06:00.986105 28666 raft_consensus.cc:2781] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:06:00.986195 28666 raft_consensus.cc:482] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:06:00.986235 28666 raft_consensus.cc:3037] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:00.987375 28666 raft_consensus.cc:504] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.987847 28666 leader_election.cc:288] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [CANDIDATE]: Term 1 election: Requested vote from peers 0cbfb4198aa14b7bb5b7083180d8713b (127.30.204.65:34563), f8522f297d6949c19e5b5d76222e4d94 (127.30.204.67:34715)
I20260807 00:06:00.988314 28545 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6504da98f491a9d5efdca46ede399" candidate_uuid: "fddf4d40ba104acbb6e0e37192698e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f8522f297d6949c19e5b5d76222e4d94"
I20260807 00:06:00.988314 28099 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "65c6504da98f491a9d5efdca46ede399" candidate_uuid: "fddf4d40ba104acbb6e0e37192698e03" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "0cbfb4198aa14b7bb5b7083180d8713b"
I20260807 00:06:00.988474 28545 raft_consensus.cc:3037] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:00.988474 28099 raft_consensus.cc:3037] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:00.989882 28545 raft_consensus.cc:2454] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fddf4d40ba104acbb6e0e37192698e03 in term 1.
I20260807 00:06:00.989882 28099 raft_consensus.cc:2454] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate fddf4d40ba104acbb6e0e37192698e03 in term 1.
I20260807 00:06:00.990331 28235 leader_election.cc:302] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 3 yes votes; 0 no votes. yes voters: 0cbfb4198aa14b7bb5b7083180d8713b, f8522f297d6949c19e5b5d76222e4d94, fddf4d40ba104acbb6e0e37192698e03; no voters:
I20260807 00:06:00.990546 28666 raft_consensus.cc:2781] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:00.990804 28666 raft_consensus.cc:686] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 1 LEADER]: Becoming Leader. State: Replica: fddf4d40ba104acbb6e0e37192698e03, State: Running, Role: LEADER
I20260807 00:06:00.991207 28666 consensus_queue.cc:237] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } }
I20260807 00:06:00.994571 27846 catalog_manager.cc:5495] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 reported cstate change: term changed from 0 to 1, leader changed from <none> to fddf4d40ba104acbb6e0e37192698e03 (127.30.204.66). New cstate: current_term: 1 leader_uuid: "fddf4d40ba104acbb6e0e37192698e03" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fddf4d40ba104acbb6e0e37192698e03" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 41841 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 } health_report { overall_health: UNKNOWN } } }
I20260807 00:06:01.008659 28648 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.004848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60842 (local address 127.30.204.66:41841)
0807 00:06:01.004993 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:01.004998 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:01.005014 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:06:01.005102 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:01.005105 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:01.005161 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:01.005258 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:01.005520 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.006104 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.007014 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.007240 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.007332 (+ 92us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.007735 (+ 403us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:01.008172 (+ 437us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:01.008314 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:01.008443 (+ 129us) server_negotiation.cc:299] Negotiation successful
0807 00:06:01.008490 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 00:06:01.012465 28631 tablet.cc:2369] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:06:01.015655 28099 raft_consensus.cc:1261] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 1 FOLLOWER]: Refusing update from remote peer fddf4d40ba104acbb6e0e37192698e03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:01.015655 28545 raft_consensus.cc:1261] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 1 FOLLOWER]: Refusing update from remote peer fddf4d40ba104acbb6e0e37192698e03: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:01.016635 28670 consensus_queue.cc:1035] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "0cbfb4198aa14b7bb5b7083180d8713b" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 34563 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:01.016878 28666 consensus_queue.cc:1035] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f8522f297d6949c19e5b5d76222e4d94" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 34715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:01.036432 27844 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:01.036618 27844 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:41814:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:01.039616 27844 catalog_manager.cc:5782] T 00000000000000000000000000000000 P a2fa5ad97a694cf48184be9ea89de194: Sending DeleteTablet for 3 replicas of tablet 65c6504da98f491a9d5efdca46ede399
I20260807 00:06:01.040597 28306 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c6504da98f491a9d5efdca46ede399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:01 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:60832
I20260807 00:06:01.040654 28035 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c6504da98f491a9d5efdca46ede399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:01 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:52140
I20260807 00:06:01.040742 28510 tablet_service.cc:1514] Processing DeleteTablet for tablet 65c6504da98f491a9d5efdca46ede399 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:01 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:60164
I20260807 00:06:01.041213 28688 tablet_replica.cc:330] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: stopping tablet replica
I20260807 00:06:01.041476 28692 tablet_replica.cc:330] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b: stopping tablet replica
I20260807 00:06:01.041757 28688 raft_consensus.cc:2229] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:06:01.041822 28692 raft_consensus.cc:2229] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:06:01.041923 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 27955
I20260807 00:06:01.042201 28692 raft_consensus.cc:2258] T 65c6504da98f491a9d5efdca46ede399 P 0cbfb4198aa14b7bb5b7083180d8713b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:01.042564 28693 tablet_replica.cc:330] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: stopping tablet replica
I20260807 00:06:01.042871 28693 raft_consensus.cc:2229] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:06:01.045140 28693 raft_consensus.cc:2258] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:01.050176 28688 raft_consensus.cc:2258] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:01.051313 28693 ts_tablet_manager.cc:1905] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:01.054868 28693 ts_tablet_manager.cc:1918] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:01.055020 28693 log.cc:1196] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/65c6504da98f491a9d5efdca46ede399
I20260807 00:06:01.055431 28693 ts_tablet_manager.cc:1939] T 65c6504da98f491a9d5efdca46ede399 P f8522f297d6949c19e5b5d76222e4d94: Deleting consensus metadata
I20260807 00:06:01.056481 28688 ts_tablet_manager.cc:1905] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:01.057219 27819 catalog_manager.cc:4841] TS f8522f297d6949c19e5b5d76222e4d94 (127.30.204.67:34715): tablet 65c6504da98f491a9d5efdca46ede399 (table test-table [id=60e87e16e74443f3a9366c7dd212701f]) successfully deleted
I20260807 00:06:01.059798 28688 ts_tablet_manager.cc:1918] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:01.060007 28688 log.cc:1196] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/65c6504da98f491a9d5efdca46ede399
I20260807 00:06:01.060474 28688 ts_tablet_manager.cc:1939] T 65c6504da98f491a9d5efdca46ede399 P fddf4d40ba104acbb6e0e37192698e03: Deleting consensus metadata
I20260807 00:06:01.062093 27817 catalog_manager.cc:4841] TS fddf4d40ba104acbb6e0e37192698e03 (127.30.204.66:41841): tablet 65c6504da98f491a9d5efdca46ede399 (table test-table [id=60e87e16e74443f3a9366c7dd212701f]) successfully deleted
W20260807 00:06:01.074900 27817 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:34563 (error 108)
I20260807 00:06:01.075623 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 28204
I20260807 00:06:01.082935 28650 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.082494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:34563 (local address 127.0.0.1:52146)
0807 00:06:01.082734 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:01.082827 (+ 93us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:34563: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150}
W20260807 00:06:01.083314 27817 catalog_manager.cc:4584] TS 0cbfb4198aa14b7bb5b7083180d8713b (127.30.204.65:34563): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 65c6504da98f491a9d5efdca46ede399: Network error: Client connection negotiation failed: client connection to 127.30.204.65:34563: connect: Connection refused (error 111)
I20260807 00:06:01.091794 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 28421
I20260807 00:06:01.104053 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 27785
2026-08-07T00:06:01Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2667 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-08-07T00:06:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:01Z Disabled control of system clock
I20260807 00:06:01.145859 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39039
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:39039
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:06:01.265405 28700 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:01.265743 28700 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:01.265796 28700 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:01.270061 28700 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:01.270148 28700 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:01.270170 28700 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:01.270192 28700 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:01.270213 28700 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:01.275372 28700 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:39039
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39039
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28700
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:01.276611 28700 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:01.278088 28700 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:01.287424 28707 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:01.287493 28700 server_base.cc:1034] running on GCE node
W20260807 00:06:01.287436 28711 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:01.287424 28709 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:01.288302 28700 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:01.289561 28700 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:01.290750 28700 hybrid_clock.cc:648] HybridClock initialized: now 1786061161290718 us; error 46 us; skew 500 ppm
I20260807 00:06:01.292789 28700 webserver.cc:460] Webserver started at http://127.30.204.126:38947/ using document root <none> and password file <none>
I20260807 00:06:01.293413 28700 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:01.293480 28700 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:01.293672 28700 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:01.296020 28700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "b87da8abfa8d42e18913f3b11e507eb9"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "8b4ee80a5c58a16cb7342cb723ae7101"
server_key_iv: "720dc8edcbee73ed0d7fabcd0a4cbe41"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.296638 28700 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "b87da8abfa8d42e18913f3b11e507eb9"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "8b4ee80a5c58a16cb7342cb723ae7101"
server_key_iv: "720dc8edcbee73ed0d7fabcd0a4cbe41"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.300958 28700 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:01.303977 28720 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.305444 28700 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:01.305603 28700 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "b87da8abfa8d42e18913f3b11e507eb9"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "8b4ee80a5c58a16cb7342cb723ae7101"
server_key_iv: "720dc8edcbee73ed0d7fabcd0a4cbe41"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.305734 28700 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:01.318953 28700 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:01.319896 28700 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:01.320076 28700 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:01.327639 28700 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:39039
I20260807 00:06:01.327801 28810 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:39039 every 8 connection(s)
I20260807 00:06:01.328711 28700 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:01.332288 28811 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:01.333710 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 28700
I20260807 00:06:01.333840 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:01.334129 31537 external_mini_cluster.cc:1428] Setting key a164c22076728b469d1e069d09845b2b
I20260807 00:06:01.339048 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9: Bootstrap starting.
I20260807 00:06:01.341562 28811 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:01.342350 28811 log.cc:824] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:01.344059 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.335800 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40118 (local address 127.30.204.126:39039)
0807 00:06:01.336623 (+ 823us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:01.336640 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:01.336682 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:06:01.337394 (+ 712us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:01.337414 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:01.337866 (+ 452us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:01.338207 (+ 341us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:01.338672 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.339761 (+ 1089us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.340683 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.341174 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.341337 (+ 163us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.342193 (+ 856us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:01.342198 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:01.342213 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:01.342651 (+ 438us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:01.342716 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:01.342740 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:01.342828 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:01.342972 (+ 144us) server_negotiation.cc:299] Negotiation successful
0807 00:06:01.343255 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":414,"thread_start_us":157,"threads_started":1}
I20260807 00:06:01.344595 28811 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9: No bootstrap required, opened a new log
I20260807 00:06:01.348460 28811 raft_consensus.cc:348] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87da8abfa8d42e18913f3b11e507eb9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39039 } }
I20260807 00:06:01.348800 28811 raft_consensus.cc:374] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:01.348868 28811 raft_consensus.cc:729] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b87da8abfa8d42e18913f3b11e507eb9, State: Initialized, Role: FOLLOWER
I20260807 00:06:01.349349 28811 consensus_queue.cc:260] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87da8abfa8d42e18913f3b11e507eb9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39039 } }
I20260807 00:06:01.349496 28811 raft_consensus.cc:388] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:01.349570 28811 raft_consensus.cc:482] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:01.349677 28811 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:01.350894 28811 raft_consensus.cc:504] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87da8abfa8d42e18913f3b11e507eb9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39039 } }
I20260807 00:06:01.351272 28811 leader_election.cc:302] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b87da8abfa8d42e18913f3b11e507eb9; no voters:
I20260807 00:06:01.351646 28811 leader_election.cc:288] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:01.351760 28817 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:01.351980 28817 raft_consensus.cc:686] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [term 1 LEADER]: Becoming Leader. State: Replica: b87da8abfa8d42e18913f3b11e507eb9, State: Running, Role: LEADER
I20260807 00:06:01.352303 28817 consensus_queue.cc:237] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87da8abfa8d42e18913f3b11e507eb9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39039 } }
I20260807 00:06:01.352958 28811 sys_catalog.cc:564] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:01.353991 28819 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b87da8abfa8d42e18913f3b11e507eb9. Latest consensus state: current_term: 1 leader_uuid: "b87da8abfa8d42e18913f3b11e507eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87da8abfa8d42e18913f3b11e507eb9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39039 } } }
I20260807 00:06:01.354133 28819 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:01.354527 28829 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:01.354512 28818 sys_catalog.cc:455] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b87da8abfa8d42e18913f3b11e507eb9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b87da8abfa8d42e18913f3b11e507eb9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39039 } } }
I20260807 00:06:01.354761 28818 sys_catalog.cc:458] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:01.357960 28829 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:01.364681 28829 catalog_manager.cc:1347] Generated new cluster ID: 107fff82d8b34bc191d20dd62a5c24fa
I20260807 00:06:01.364780 28829 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:01.378611 28829 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:01.379724 28829 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:01.391296 28829 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9: Generated new TSK 0
I20260807 00:06:01.392334 28829 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:01.397819 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39039
--builtin_ntp_servers=127.30.204.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:06:01.518424 28846 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:01.518735 28846 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:01.518775 28846 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:01.522994 28846 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:01.523082 28846 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:01.523146 28846 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:06:01.528342 28846 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28846
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:01.529537 28846 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:01.531080 28846 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:01.541590 28862 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:01.541595 28858 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:01.541637 28857 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:01.541832 28846 server_base.cc:1034] running on GCE node
I20260807 00:06:01.542356 28846 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:01.543013 28846 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:01.544167 28846 hybrid_clock.cc:648] HybridClock initialized: now 1786061161544152 us; error 39 us; skew 500 ppm
I20260807 00:06:01.546068 28846 webserver.cc:460] Webserver started at http://127.30.204.65:35307/ using document root <none> and password file <none>
I20260807 00:06:01.546641 28846 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:01.546705 28846 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:01.546888 28846 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:01.549052 28846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "f2a58ab895374a47800303fcc7871478"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "a0b31085ea26408344dad03154dc9623"
server_key_iv: "9ccb5ac510d5ddc0dd3d482a013147a4"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.549611 28846 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "f2a58ab895374a47800303fcc7871478"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "a0b31085ea26408344dad03154dc9623"
server_key_iv: "9ccb5ac510d5ddc0dd3d482a013147a4"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.553722 28846 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:06:01.556344 28870 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.557597 28846 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 00:06:01.557719 28846 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "f2a58ab895374a47800303fcc7871478"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "a0b31085ea26408344dad03154dc9623"
server_key_iv: "9ccb5ac510d5ddc0dd3d482a013147a4"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.557821 28846 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:01.570015 28846 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:01.570782 28846 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:01.570950 28846 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:01.571671 28846 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:01.572863 28846 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:01.572916 28846 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.572958 28846 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:01.572971 28846 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.582067 28846 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:36757
I20260807 00:06:01.582106 29053 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:36757 every 8 connection(s)
I20260807 00:06:01.583098 28846 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:06:01.585736 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 28846
I20260807 00:06:01.585865 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:06:01.586189 31537 external_mini_cluster.cc:1428] Setting key 8a993aafc00c6aa96ef0fa1b7ef6bc09
I20260807 00:06:01.588732 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39039
--builtin_ntp_servers=127.30.204.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:06:01.593954 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.585132 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:46103 (local address 127.30.204.126:39039)
0807 00:06:01.585275 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:01.585279 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:01.586520 (+ 1241us) server_negotiation.cc:407] Connection header received
0807 00:06:01.588301 (+ 1781us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:01.588306 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:01.588364 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:01.588444 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:01.589747 (+ 1303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.590318 (+ 571us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.591672 (+ 1354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.591997 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.592119 (+ 122us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.593326 (+ 1207us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:01.593330 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:01.593333 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:01.593353 (+ 20us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:01.593381 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:01.593394 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:01.593486 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:01.593690 (+ 204us) server_negotiation.cc:299] Negotiation successful
0807 00:06:01.593778 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260807 00:06:01.595044 29058 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.585481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39039 (local address 127.30.204.65:46103)
0807 00:06:01.586318 (+ 837us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:01.586380 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:01.587992 (+ 1612us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:01.588688 (+ 696us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:01.588699 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:01.588833 (+ 134us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:01.589565 (+ 732us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:01.589586 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.590464 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.590468 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:01.591534 (+ 1066us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:01.591545 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.592155 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.592157 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:01.592335 (+ 178us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.592982 (+ 647us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:06:01.592984 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:01.593032 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:06:01.593129 (+ 97us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:01.593511 (+ 382us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:01.593522 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:01.593528 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:06:01.593665 (+ 137us) client_negotiation.cc:240] Negotiation successful
0807 00:06:01.593985 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":308,"thread_start_us":133,"threads_started":1}
I20260807 00:06:01.596920 29054 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39039
I20260807 00:06:01.597258 29054 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:01.598027 29054 heartbeater.cc:511] Master 127.30.204.126:39039 requested a full tablet report, sending...
I20260807 00:06:01.599996 28753 ts_manager.cc:194] Registered new tserver with Master: f2a58ab895374a47800303fcc7871478 (127.30.204.65:36757)
I20260807 00:06:01.601222 28753 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.65:46103
W20260807 00:06:01.708184 29059 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:01.708483 29059 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:01.708523 29059 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:01.712764 29059 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:01.712838 29059 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:01.712895 29059 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:06:01.717872 29059 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29059
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:01.719092 29059 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:01.720568 29059 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:01.730561 29066 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:01.730561 29067 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:01.730620 29069 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:01.730981 29059 server_base.cc:1034] running on GCE node
I20260807 00:06:01.731424 29059 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:01.732162 29059 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:01.733332 29059 hybrid_clock.cc:648] HybridClock initialized: now 1786061161733296 us; error 47 us; skew 500 ppm
I20260807 00:06:01.735375 29059 webserver.cc:460] Webserver started at http://127.30.204.66:44867/ using document root <none> and password file <none>
I20260807 00:06:01.736034 29059 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:01.736109 29059 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:01.736344 29059 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:01.738627 29059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "e0dee7da1a0f414998e07569a4510218"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "9d3591e5f9b0503b9b2ac958977845f8"
server_key_iv: "8af2fd5bbfb3676531a2afdca05a8187"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.739230 29059 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "e0dee7da1a0f414998e07569a4510218"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "9d3591e5f9b0503b9b2ac958977845f8"
server_key_iv: "8af2fd5bbfb3676531a2afdca05a8187"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.743225 29059 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:06:01.745975 29080 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.747625 29059 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:01.747758 29059 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "e0dee7da1a0f414998e07569a4510218"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "9d3591e5f9b0503b9b2ac958977845f8"
server_key_iv: "8af2fd5bbfb3676531a2afdca05a8187"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.747880 29059 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:01.765497 29059 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:01.766306 29059 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:01.766513 29059 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:01.767213 29059 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:01.768342 29059 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:01.768393 29059 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.768451 29059 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:01.768481 29059 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.778047 29059 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:33311
I20260807 00:06:01.778071 29271 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:33311 every 8 connection(s)
I20260807 00:06:01.779018 29059 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:06:01.787243 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 29059
I20260807 00:06:01.787377 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:06:01.787721 31537 external_mini_cluster.cc:1428] Setting key b71fbbcfd39a7a11b100e372bd526fd2
I20260807 00:06:01.788924 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.780947 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:39283 (local address 127.30.204.126:39039)
0807 00:06:01.781102 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:01.781106 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:01.782278 (+ 1172us) server_negotiation.cc:407] Connection header received
0807 00:06:01.783844 (+ 1566us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:01.783847 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:01.783913 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:01.783985 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:01.785000 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.785534 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.786731 (+ 1197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.786960 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.787056 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.788363 (+ 1307us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:01.788367 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:01.788369 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:01.788383 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:01.788405 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:01.788414 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:01.788473 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:01.788698 (+ 225us) server_negotiation.cc:299] Negotiation successful
0807 00:06:01.788777 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:06:01.789829 29282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.781394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39039 (local address 127.30.204.66:39283)
0807 00:06:01.782101 (+ 707us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:01.782147 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:01.783575 (+ 1428us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:01.784190 (+ 615us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:01.784221 (+ 31us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:01.784318 (+ 97us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:01.784843 (+ 525us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:01.784858 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.785666 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.785669 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:01.786593 (+ 924us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:01.786604 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.787125 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.787130 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:01.787310 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.788031 (+ 721us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:06:01.788032 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:01.788080 (+ 48us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:06:01.788166 (+ 86us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:01.788494 (+ 328us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:01.788505 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:01.788513 (+ 8us) client_negotiation.cc:769] Sending connection context
0807 00:06:01.788652 (+ 139us) client_negotiation.cc:240] Negotiation successful
0807 00:06:01.788925 (+ 273us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":119,"threads_started":1}
I20260807 00:06:01.790026 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39039
--builtin_ntp_servers=127.30.204.84:41719
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:06:01.791010 29273 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39039
I20260807 00:06:01.791270 29273 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:01.791887 29273 heartbeater.cc:511] Master 127.30.204.126:39039 requested a full tablet report, sending...
I20260807 00:06:01.792971 28753 ts_manager.cc:194] Registered new tserver with Master: e0dee7da1a0f414998e07569a4510218 (127.30.204.66:33311)
I20260807 00:06:01.793557 28753 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.66:39283
W20260807 00:06:01.910053 29288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:01.910377 29288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:01.910418 29288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:01.914669 29288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:01.914762 29288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:01.914819 29288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:06:01.920028 29288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41719
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39039
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:01.921197 29288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:01.922658 29288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:01.934911 29303 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:01.935114 29288 server_base.cc:1034] running on GCE node
W20260807 00:06:01.934913 29301 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:01.934952 29300 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:01.935755 29288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:01.936565 29288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:01.937774 29288 hybrid_clock.cc:648] HybridClock initialized: now 1786061161937755 us; error 35 us; skew 500 ppm
I20260807 00:06:01.939970 29288 webserver.cc:460] Webserver started at http://127.30.204.67:39217/ using document root <none> and password file <none>
I20260807 00:06:01.940605 29288 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:01.940699 29288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:01.940932 29288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:01.943208 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "475ac4826e33444295fc3fffa5cfafdb"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "3af0f49c1d95d4b48c1c8c5942dd945f"
server_key_iv: "2cccb2fa645e87bbe5ac197835cc561e"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.943867 29288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "475ac4826e33444295fc3fffa5cfafdb"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "3af0f49c1d95d4b48c1c8c5942dd945f"
server_key_iv: "2cccb2fa645e87bbe5ac197835cc561e"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.947893 29288 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:01.950592 29315 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.952061 29288 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:01.952194 29288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "475ac4826e33444295fc3fffa5cfafdb"
format_stamp: "Formatted at 2026-08-07 00:06:01 on dist-test-slave-pww0"
server_key: "3af0f49c1d95d4b48c1c8c5942dd945f"
server_key_iv: "2cccb2fa645e87bbe5ac197835cc561e"
server_key_version: "encryptionkey@0"
I20260807 00:06:01.952328 29288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:01.970319 29288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:01.971122 29288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:01.971329 29288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:01.972043 29288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:01.973088 29288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:01.973138 29288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.973196 29288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:01.973225 29288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:01.982473 29288 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:35471
I20260807 00:06:01.982517 29494 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:35471 every 8 connection(s)
I20260807 00:06:01.983566 29288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:06:01.988315 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 29288
I20260807 00:06:01.988433 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:06:01.988724 31537 external_mini_cluster.cc:1428] Setting key 10dadeb637bffe9ea636a67368f7be75
I20260807 00:06:01.994166 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.985686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:57363 (local address 127.30.204.126:39039)
0807 00:06:01.985874 (+ 188us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:01.985879 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:01.987077 (+ 1198us) server_negotiation.cc:407] Connection header received
0807 00:06:01.988841 (+ 1764us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:01.988845 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:01.988911 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:01.989015 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:01.990231 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.990815 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.992158 (+ 1343us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.992396 (+ 238us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.992496 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.993712 (+ 1216us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:01.993715 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:01.993718 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:01.993752 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:01.993776 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:01.993785 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:01.993847 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:01.993974 (+ 127us) server_negotiation.cc:299] Negotiation successful
0807 00:06:01.994040 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":83}
I20260807 00:06:01.995139 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:01.986119 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39039 (local address 127.30.204.67:57363)
0807 00:06:01.986908 (+ 789us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:01.986955 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:01.988561 (+ 1606us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:01.989215 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:01.989226 (+ 11us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:01.989347 (+ 121us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:01.990078 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:01.990103 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.990957 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.990960 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:01.992012 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:01.992024 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:01.992571 (+ 547us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:01.992574 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:01.992754 (+ 180us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:01.993414 (+ 660us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:06:01.993416 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:01.993466 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:06:01.993545 (+ 79us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:01.993857 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:01.993862 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:01.993867 (+ 5us) client_negotiation.cc:769] Sending connection context
0807 00:06:01.993966 (+ 99us) client_negotiation.cc:240] Negotiation successful
0807 00:06:01.994233 (+ 267us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328,"thread_start_us":136,"threads_started":1}
I20260807 00:06:01.996498 29496 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39039
I20260807 00:06:01.996832 29496 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:01.997577 29496 heartbeater.cc:511] Master 127.30.204.126:39039 requested a full tablet report, sending...
I20260807 00:06:01.998824 28753 ts_manager.cc:194] Registered new tserver with Master: 475ac4826e33444295fc3fffa5cfafdb (127.30.204.67:35471)
I20260807 00:06:01.999614 28753 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.67:57363
I20260807 00:06:02.007036 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:06:02.013566 28815 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.009542 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57914 (local address 127.30.204.126:39039)
0807 00:06:02.009712 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:02.009717 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:02.009835 (+ 118us) server_negotiation.cc:407] Connection header received
0807 00:06:02.009957 (+ 122us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:02.009960 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:02.010017 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:02.010101 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:02.010415 (+ 314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.010954 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.011881 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.012108 (+ 227us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.012198 (+ 90us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.013097 (+ 899us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:02.013100 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:02.013102 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:02.013117 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:02.013138 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:02.013146 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:02.013261 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:02.013361 (+ 100us) server_negotiation.cc:299] Negotiation successful
0807 00:06:02.013399 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 00:06:02.016959 28753 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:57914:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:06:02.020219 28753 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:06:02.036523 29522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.030327 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:35471 (local address 127.0.0.1:48456)
0807 00:06:02.031171 (+ 844us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:02.031186 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:02.031290 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:02.031728 (+ 438us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:02.031731 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:02.031750 (+ 19us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:02.031847 (+ 97us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.031853 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.032911 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.032915 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.034423 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.034436 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.035843 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.035847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.036024 (+ 177us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.036087 (+ 63us) client_negotiation.cc:769] Sending connection context
0807 00:06:02.036202 (+ 115us) client_negotiation.cc:240] Negotiation successful
0807 00:06:02.036351 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":777,"mutex_wait_us":78,"thread_start_us":100,"threads_started":1}
I20260807 00:06:02.036576 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.029958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59242 (local address 127.30.204.66:33311)
0807 00:06:02.030371 (+ 413us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:02.030384 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:02.030870 (+ 486us) server_negotiation.cc:407] Connection header received
0807 00:06:02.031068 (+ 198us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:02.031073 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:02.031260 (+ 187us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:02.031384 (+ 124us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:02.031805 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.033045 (+ 1240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.034686 (+ 1641us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.035435 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.035565 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.035774 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:02.036285 (+ 511us) server_negotiation.cc:299] Negotiation successful
0807 00:06:02.036398 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":319,"thread_start_us":99,"threads_started":1}
I20260807 00:06:02.036821 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.029697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53148 (local address 127.30.204.65:36757)
0807 00:06:02.030594 (+ 897us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:02.030606 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:02.030632 (+ 26us) server_negotiation.cc:407] Connection header received
0807 00:06:02.030832 (+ 200us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:02.030839 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:02.031067 (+ 228us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:02.031211 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:02.031786 (+ 575us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.033045 (+ 1259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.035553 (+ 2508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.036095 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.036199 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.036422 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:02.036473 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 00:06:02.036566 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":793,"thread_start_us":163,"threads_started":1}
I20260807 00:06:02.037125 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.030746 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48456 (local address 127.30.204.67:35471)
0807 00:06:02.031242 (+ 496us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:02.031255 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:02.031277 (+ 22us) server_negotiation.cc:407] Connection header received
0807 00:06:02.031413 (+ 136us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:02.031417 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:02.031608 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:02.031752 (+ 144us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:02.031951 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.032780 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.034570 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.035435 (+ 865us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.035565 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.035774 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:02.036798 (+ 1024us) server_negotiation.cc:299] Negotiation successful
0807 00:06:02.036965 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":404,"thread_start_us":150,"threads_started":1}
I20260807 00:06:02.037087 29520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.029538 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36757 (local address 127.0.0.1:53148)
0807 00:06:02.030418 (+ 880us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:02.030480 (+ 62us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:02.030620 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:02.031400 (+ 780us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:02.031407 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:02.031430 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:02.031621 (+ 191us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.031628 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.033182 (+ 1554us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.033185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.034563 (+ 1378us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.034575 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.036221 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.036225 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.036327 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.036350 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:06:02.036412 (+ 62us) client_negotiation.cc:240] Negotiation successful
0807 00:06:02.036493 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":672,"mutex_wait_us":53,"thread_start_us":94,"threads_started":1}
I20260807 00:06:02.037218 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.029793 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:33311 (local address 127.0.0.1:59242)
0807 00:06:02.030775 (+ 982us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:02.030788 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:02.030936 (+ 148us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:02.031597 (+ 661us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:02.031600 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:02.031616 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:02.031685 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.031691 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.033182 (+ 1491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.033185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.034564 (+ 1379us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.034575 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.035843 (+ 1268us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.035847 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.036028 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.036086 (+ 58us) client_negotiation.cc:769] Sending connection context
0807 00:06:02.037026 (+ 940us) client_negotiation.cc:240] Negotiation successful
0807 00:06:02.037107 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":919,"mutex_wait_us":123,"thread_start_us":114,"threads_started":1}
I20260807 00:06:02.040345 29161 tablet_service.cc:1467] Processing CreateTablet for tablet 1e6d46004d1245caa8333c4f7f2ed90b (DEFAULT_TABLE table=test-table [id=4cca9dc5209649f1a07e6bbb5bc86966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:02.040508 28945 tablet_service.cc:1467] Processing CreateTablet for tablet 1e6d46004d1245caa8333c4f7f2ed90b (DEFAULT_TABLE table=test-table [id=4cca9dc5209649f1a07e6bbb5bc86966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:02.040455 29389 tablet_service.cc:1467] Processing CreateTablet for tablet 1e6d46004d1245caa8333c4f7f2ed90b (DEFAULT_TABLE table=test-table [id=4cca9dc5209649f1a07e6bbb5bc86966]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:02.041541 28945 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e6d46004d1245caa8333c4f7f2ed90b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:02.041541 29389 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e6d46004d1245caa8333c4f7f2ed90b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:02.041535 29161 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 1e6d46004d1245caa8333c4f7f2ed90b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:02.048372 29524 tablet_bootstrap.cc:492] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Bootstrap starting.
I20260807 00:06:02.049999 29526 tablet_bootstrap.cc:492] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Bootstrap starting.
I20260807 00:06:02.050797 29524 tablet_bootstrap.cc:654] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:02.051608 29524 log.cc:824] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:02.051663 29527 tablet_bootstrap.cc:492] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: Bootstrap starting.
I20260807 00:06:02.053625 29526 tablet_bootstrap.cc:654] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:02.053946 29527 tablet_bootstrap.cc:654] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:02.054267 29524 tablet_bootstrap.cc:492] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: No bootstrap required, opened a new log
I20260807 00:06:02.054495 29524 ts_tablet_manager.cc:1397] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.004s
I20260807 00:06:02.054680 29526 log.cc:824] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:02.054767 29527 log.cc:824] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:02.056823 29527 tablet_bootstrap.cc:492] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: No bootstrap required, opened a new log
I20260807 00:06:02.057021 29527 ts_tablet_manager.cc:1397] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:06:02.057019 29526 tablet_bootstrap.cc:492] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: No bootstrap required, opened a new log
I20260807 00:06:02.057214 29526 ts_tablet_manager.cc:1397] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:06:02.058945 29524 raft_consensus.cc:348] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.059263 29524 raft_consensus.cc:374] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:02.059325 29524 raft_consensus.cc:729] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e0dee7da1a0f414998e07569a4510218, State: Initialized, Role: FOLLOWER
I20260807 00:06:02.059875 29524 consensus_queue.cc:260] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.060612 29527 raft_consensus.cc:348] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.060954 29527 raft_consensus.cc:374] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:02.061020 29527 raft_consensus.cc:729] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2a58ab895374a47800303fcc7871478, State: Initialized, Role: FOLLOWER
I20260807 00:06:02.061066 29524 ts_tablet_manager.cc:1428] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Time spent starting tablet: real 0.007s user 0.007s sys 0.000s
I20260807 00:06:02.061468 29273 heartbeater.cc:503] Master 127.30.204.126:39039 was elected leader, sending a full tablet report...
I20260807 00:06:02.061483 29527 consensus_queue.cc:260] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.061792 29526 raft_consensus.cc:348] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.062018 29526 raft_consensus.cc:374] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:02.062050 29526 raft_consensus.cc:729] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 475ac4826e33444295fc3fffa5cfafdb, State: Initialized, Role: FOLLOWER
I20260807 00:06:02.062415 29526 consensus_queue.cc:260] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.062502 29527 ts_tablet_manager.cc:1428] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 00:06:02.062861 29054 heartbeater.cc:503] Master 127.30.204.126:39039 was elected leader, sending a full tablet report...
I20260807 00:06:02.063407 29526 ts_tablet_manager.cc:1428] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:06:02.063798 29496 heartbeater.cc:503] Master 127.30.204.126:39039 was elected leader, sending a full tablet report...
W20260807 00:06:02.084515 29055 tablet.cc:2369] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:06:02.234684 29497 tablet.cc:2369] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:06:02.280476 29275 tablet.cc:2369] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:06:02.319451 29537 raft_consensus.cc:482] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:06:02.319703 29537 raft_consensus.cc:504] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.320968 29537 leader_election.cc:288] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2a58ab895374a47800303fcc7871478 (127.30.204.65:36757), e0dee7da1a0f414998e07569a4510218 (127.30.204.66:33311)
I20260807 00:06:02.325084 29521 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.321426 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:46595 (local address 127.30.204.66:33311)
0807 00:06:02.321602 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:02.321607 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:02.321632 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:06:02.321822 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:02.321826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:02.321876 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:02.321967 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:02.322326 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.322861 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.324019 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.324501 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.324615 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.324677 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:02.324896 (+ 219us) server_negotiation.cc:299] Negotiation successful
0807 00:06:02.324973 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":80}
I20260807 00:06:02.325094 29499 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.321308 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:33311 (local address 127.30.204.67:46595)
0807 00:06:02.321531 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:02.321548 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:02.321670 (+ 122us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:02.322013 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:02.322016 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:02.322037 (+ 21us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:02.322207 (+ 170us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.322214 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.322974 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.322977 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.323875 (+ 898us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.323887 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.324634 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.324638 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.324753 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.324776 (+ 23us) client_negotiation.cc:769] Sending connection context
0807 00:06:02.324854 (+ 78us) client_negotiation.cc:240] Negotiation successful
0807 00:06:02.324941 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":20}
I20260807 00:06:02.325274 29539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.321334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36757 (local address 127.30.204.67:35271)
0807 00:06:02.321766 (+ 432us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:02.321778 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:02.321871 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:02.322089 (+ 218us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:02.322092 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:02.322108 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:02.322272 (+ 164us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.322278 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.322979 (+ 701us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.322982 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.323902 (+ 920us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:02.323913 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.324900 (+ 987us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.324904 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:02.324991 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.325012 (+ 21us) client_negotiation.cc:769] Sending connection context
0807 00:06:02.325093 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:06:02.325157 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"mutex_wait_us":18,"thread_start_us":145,"threads_started":1}
I20260807 00:06:02.325345 29519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.321419 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:35271 (local address 127.30.204.65:36757)
0807 00:06:02.321552 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:02.321556 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:02.321858 (+ 302us) server_negotiation.cc:407] Connection header received
0807 00:06:02.321955 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:02.321958 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:02.322004 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:02.322066 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:02.322385 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.322874 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.324088 (+ 1214us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.324608 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.324726 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.324802 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:02.325133 (+ 331us) server_negotiation.cc:299] Negotiation successful
0807 00:06:02.325226 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260807 00:06:02.325932 28977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6d46004d1245caa8333c4f7f2ed90b" candidate_uuid: "475ac4826e33444295fc3fffa5cfafdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a58ab895374a47800303fcc7871478" is_pre_election: true
I20260807 00:06:02.325923 29189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6d46004d1245caa8333c4f7f2ed90b" candidate_uuid: "475ac4826e33444295fc3fffa5cfafdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0dee7da1a0f414998e07569a4510218" is_pre_election: true
I20260807 00:06:02.326277 29189 raft_consensus.cc:2454] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475ac4826e33444295fc3fffa5cfafdb in term 0.
I20260807 00:06:02.326280 28977 raft_consensus.cc:2454] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 475ac4826e33444295fc3fffa5cfafdb in term 0.
I20260807 00:06:02.326828 29320 leader_election.cc:302] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475ac4826e33444295fc3fffa5cfafdb, f2a58ab895374a47800303fcc7871478; no voters:
I20260807 00:06:02.327159 29537 raft_consensus.cc:2781] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:06:02.327255 29537 raft_consensus.cc:482] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:06:02.327332 29537 raft_consensus.cc:3037] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:02.328482 29537 raft_consensus.cc:504] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.328887 29537 leader_election.cc:288] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [CANDIDATE]: Term 1 election: Requested vote from peers f2a58ab895374a47800303fcc7871478 (127.30.204.65:36757), e0dee7da1a0f414998e07569a4510218 (127.30.204.66:33311)
I20260807 00:06:02.329353 28977 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6d46004d1245caa8333c4f7f2ed90b" candidate_uuid: "475ac4826e33444295fc3fffa5cfafdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2a58ab895374a47800303fcc7871478"
I20260807 00:06:02.329491 28977 raft_consensus.cc:3037] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:02.330641 28977 raft_consensus.cc:2454] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475ac4826e33444295fc3fffa5cfafdb in term 1.
I20260807 00:06:02.331032 29320 leader_election.cc:302] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 475ac4826e33444295fc3fffa5cfafdb, f2a58ab895374a47800303fcc7871478; no voters:
I20260807 00:06:02.330978 29189 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "1e6d46004d1245caa8333c4f7f2ed90b" candidate_uuid: "475ac4826e33444295fc3fffa5cfafdb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e0dee7da1a0f414998e07569a4510218"
I20260807 00:06:02.331164 29189 raft_consensus.cc:3037] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:02.331238 29537 raft_consensus.cc:2781] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:02.331485 29537 raft_consensus.cc:686] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 1 LEADER]: Becoming Leader. State: Replica: 475ac4826e33444295fc3fffa5cfafdb, State: Running, Role: LEADER
I20260807 00:06:02.331882 29537 consensus_queue.cc:237] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } }
I20260807 00:06:02.332624 29189 raft_consensus.cc:2454] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 475ac4826e33444295fc3fffa5cfafdb in term 1.
I20260807 00:06:02.335691 28747 catalog_manager.cc:5495] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb reported cstate change: term changed from 0 to 1, leader changed from <none> to 475ac4826e33444295fc3fffa5cfafdb (127.30.204.67). New cstate: current_term: 1 leader_uuid: "475ac4826e33444295fc3fffa5cfafdb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "475ac4826e33444295fc3fffa5cfafdb" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 35471 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 } health_report { overall_health: UNKNOWN } } }
I20260807 00:06:02.406611 29523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.402751 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48460 (local address 127.30.204.67:35471)
0807 00:06:02.402870 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:02.402874 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:02.402889 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:06:02.402973 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:02.402975 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:02.403043 (+ 68us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:02.403137 (+ 94us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:02.403522 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.404070 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.405029 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:02.405272 (+ 243us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:02.405358 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:02.405739 (+ 381us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:02.406182 (+ 443us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:02.406300 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:02.406428 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 00:06:02.406467 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260807 00:06:02.413537 28977 raft_consensus.cc:1261] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 1 FOLLOWER]: Refusing update from remote peer 475ac4826e33444295fc3fffa5cfafdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:02.413538 29189 raft_consensus.cc:1261] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 1 FOLLOWER]: Refusing update from remote peer 475ac4826e33444295fc3fffa5cfafdb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:02.414440 29540 consensus_queue.cc:1035] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2a58ab895374a47800303fcc7871478" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 36757 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:02.414649 29537 consensus_queue.cc:1035] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [LEADER]: Connected to new peer: Peer: permanent_uuid: "e0dee7da1a0f414998e07569a4510218" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 33311 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:02.421844 29543 mvcc.cc:205] Tried to move back new op lower bound from 7315706521236795392 to 7315706520915017728. Current Snapshot: MvccSnapshot[applied={T|T < 7315706521236795392}]
I20260807 00:06:02.423395 29545 mvcc.cc:205] Tried to move back new op lower bound from 7315706521236795392 to 7315706520915017728. Current Snapshot: MvccSnapshot[applied={T|T < 7315706521236795392}]
I20260807 00:06:02.432354 28747 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:57914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:02.432538 28747 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:57914:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:02.435341 28747 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b87da8abfa8d42e18913f3b11e507eb9: Sending DeleteTablet for 3 replicas of tablet 1e6d46004d1245caa8333c4f7f2ed90b
I20260807 00:06:02.436401 28945 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e6d46004d1245caa8333c4f7f2ed90b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:02 UTC) from {username='slave'} at 127.0.0.1:53148
I20260807 00:06:02.436401 29161 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e6d46004d1245caa8333c4f7f2ed90b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:02 UTC) from {username='slave'} at 127.0.0.1:59242
I20260807 00:06:02.436724 29389 tablet_service.cc:1514] Processing DeleteTablet for tablet 1e6d46004d1245caa8333c4f7f2ed90b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:02 UTC) from {username='slave'} at 127.0.0.1:48456
I20260807 00:06:02.437307 29555 tablet_replica.cc:330] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: stopping tablet replica
I20260807 00:06:02.437343 29554 tablet_replica.cc:330] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478: stopping tablet replica
I20260807 00:06:02.437608 29556 tablet_replica.cc:330] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: stopping tablet replica
I20260807 00:06:02.437704 29555 raft_consensus.cc:2229] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:06:02.437799 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 28846
I20260807 00:06:02.438019 29554 raft_consensus.cc:2229] T 1e6d46004d1245caa8333c4f7f2ed90b P f2a58ab895374a47800303fcc7871478 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:06:02.438077 29555 raft_consensus.cc:2258] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:02.438365 29556 raft_consensus.cc:2229] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:06:02.438807 29556 raft_consensus.cc:2258] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:02.442493 29556 ts_tablet_manager.cc:1905] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:02.442494 29555 ts_tablet_manager.cc:1905] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:02.445617 29555 ts_tablet_manager.cc:1918] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:02.445741 29555 log.cc:1196] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/1e6d46004d1245caa8333c4f7f2ed90b
I20260807 00:06:02.445813 29556 ts_tablet_manager.cc:1918] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:02.445933 29556 log.cc:1196] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/1e6d46004d1245caa8333c4f7f2ed90b
I20260807 00:06:02.446060 29555 ts_tablet_manager.cc:1939] T 1e6d46004d1245caa8333c4f7f2ed90b P e0dee7da1a0f414998e07569a4510218: Deleting consensus metadata
I20260807 00:06:02.446394 29556 ts_tablet_manager.cc:1939] T 1e6d46004d1245caa8333c4f7f2ed90b P 475ac4826e33444295fc3fffa5cfafdb: Deleting consensus metadata
I20260807 00:06:02.447908 28724 catalog_manager.cc:4841] TS 475ac4826e33444295fc3fffa5cfafdb (127.30.204.67:35471): tablet 1e6d46004d1245caa8333c4f7f2ed90b (table test-table [id=4cca9dc5209649f1a07e6bbb5bc86966]) successfully deleted
I20260807 00:06:02.448122 28726 catalog_manager.cc:4841] TS e0dee7da1a0f414998e07569a4510218 (127.30.204.66:33311): tablet 1e6d46004d1245caa8333c4f7f2ed90b (table test-table [id=4cca9dc5209649f1a07e6bbb5bc86966]) successfully deleted
W20260807 00:06:02.457275 28724 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:36757 (error 108)
I20260807 00:06:02.457383 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 29059
I20260807 00:06:02.469321 29518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:02.468951 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:36757 (local address 127.0.0.1:53154)
0807 00:06:02.469110 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:02.469211 (+ 101us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:36757: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260807 00:06:02.469699 28724 catalog_manager.cc:4584] TS f2a58ab895374a47800303fcc7871478 (127.30.204.65:36757): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 1e6d46004d1245caa8333c4f7f2ed90b: Network error: Client connection negotiation failed: client connection to 127.30.204.65:36757: connect: Connection refused (error 111)
I20260807 00:06:02.477586 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 29288
I20260807 00:06:02.490202 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 28700
2026-08-07T00:06:02Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1386 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:02 dist-test-slave-pww0 krb5kdc[29563](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:02 dist-test-slave-pww0 krb5kdc[29563](info): set up 2 sockets
Aug 07 00:06:02 dist-test-slave-pww0 krb5kdc[29563](info): commencing operation
krb5kdc: starting...
W20260807 00:06:03.661365 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.005s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:03 dist-test-slave-pww0 krb5kdc[29563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:03Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:03Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:03.794080 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40203
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:40203
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:03.914126 29584 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:03.914453 29584 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:03.914525 29584 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:03.918694 29584 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:03.918780 29584 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:03.918802 29584 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:03.918820 29584 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:03.918840 29584 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:03.924705 29584 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:40203
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40203
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29584
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:03.926003 29584 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:03.927119 29584 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:03.936286 29595 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:03.936281 29592 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:03.936896 29593 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:03.938134 29584 server_base.cc:1034] running on GCE node
I20260807 00:06:03.938766 29584 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:03.939993 29584 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:03.941179 29584 hybrid_clock.cc:648] HybridClock initialized: now 1786061163941167 us; error 37 us; skew 500 ppm
Aug 07 00:06:03 dist-test-slave-pww0 krb5kdc[29563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061163, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:03.944130 29584 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:03.945091 29584 webserver.cc:460] Webserver started at http://127.30.204.126:44737/ using document root <none> and password file <none>
I20260807 00:06:03.945681 29584 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:03.945756 29584 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:03.945966 29584 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:03.948233 29584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "e91a7ffff4f3439ab5f58e9768ff3da9"
format_stamp: "Formatted at 2026-08-07 00:06:03 on dist-test-slave-pww0"
server_key: "fc365a1b7dd85c9782ebe7fd45153561"
server_key_iv: "e63ff086fe50c6ee525ef329381c80eb"
server_key_version: "encryptionkey@0"
I20260807 00:06:03.948829 29584 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "e91a7ffff4f3439ab5f58e9768ff3da9"
format_stamp: "Formatted at 2026-08-07 00:06:03 on dist-test-slave-pww0"
server_key: "fc365a1b7dd85c9782ebe7fd45153561"
server_key_iv: "e63ff086fe50c6ee525ef329381c80eb"
server_key_version: "encryptionkey@0"
I20260807 00:06:03.953011 29584 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:03.956079 29612 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:03.957464 29584 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:03.957608 29584 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "e91a7ffff4f3439ab5f58e9768ff3da9"
format_stamp: "Formatted at 2026-08-07 00:06:03 on dist-test-slave-pww0"
server_key: "fc365a1b7dd85c9782ebe7fd45153561"
server_key_iv: "e63ff086fe50c6ee525ef329381c80eb"
server_key_version: "encryptionkey@0"
I20260807 00:06:03.957749 29584 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:03.973783 29584 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:03.977335 29584 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:03.977538 29584 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:03.985240 29584 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:40203
I20260807 00:06:03.985270 29702 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:40203 every 8 connection(s)
I20260807 00:06:03.986284 29584 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:03.989347 29703 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:03.992506 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 29584
I20260807 00:06:03.992662 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:03.993006 31537 external_mini_cluster.cc:1428] Setting key d61c703157f276bda8c1cdd76f3f1f4b
I20260807 00:06:03.996465 29703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9: Bootstrap starting.
I20260807 00:06:03.998891 29703 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:03.999929 29703 log.cc:824] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:04.002650 29703 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9: No bootstrap required, opened a new log
Aug 07 00:06:03 dist-test-slave-pww0 krb5kdc[29563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061163, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:04.006003 29703 raft_consensus.cc:348] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40203 } }
I20260807 00:06:04.006297 29703 raft_consensus.cc:374] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:04.006361 29703 raft_consensus.cc:729] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e91a7ffff4f3439ab5f58e9768ff3da9, State: Initialized, Role: FOLLOWER
I20260807 00:06:04.006950 29703 consensus_queue.cc:260] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40203 } }
I20260807 00:06:04.007179 29703 raft_consensus.cc:388] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:04.007265 29703 raft_consensus.cc:482] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:04.007373 29703 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:04.008430 29703 raft_consensus.cc:504] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40203 } }
I20260807 00:06:04.008828 29703 leader_election.cc:302] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e91a7ffff4f3439ab5f58e9768ff3da9; no voters:
I20260807 00:06:04.009181 29703 leader_election.cc:288] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:04.009280 29714 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:04.009523 29714 raft_consensus.cc:686] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [term 1 LEADER]: Becoming Leader. State: Replica: e91a7ffff4f3439ab5f58e9768ff3da9, State: Running, Role: LEADER
I20260807 00:06:04.009857 29714 consensus_queue.cc:237] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40203 } }
I20260807 00:06:04.010061 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:03.994772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49948 (local address 127.30.204.126:40203)
0807 00:06:03.995316 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:03.995332 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:03.995373 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:06:03.996141 (+ 768us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:03.996163 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:03.996563 (+ 400us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:03.996963 (+ 400us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:03.997845 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:03.998901 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.000172 (+ 1271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.000846 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.000997 (+ 151us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.003883 (+ 2886us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:04.003889 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:04.003904 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:04.003938 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:04.006295 (+ 2357us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.006915 (+ 620us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.006920 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.006925 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.007074 (+ 149us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.007399 (+ 325us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.007403 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.007404 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.007924 (+ 520us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:04.008148 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.008481 (+ 333us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.008870 (+ 389us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":377,"thread_start_us":166,"threads_started":1}
I20260807 00:06:04.010600 29703 sys_catalog.cc:564] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:04.011252 29714 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e91a7ffff4f3439ab5f58e9768ff3da9. Latest consensus state: current_term: 1 leader_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40203 } } }
I20260807 00:06:04.011381 29714 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:04.011745 29715 sys_catalog.cc:455] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e91a7ffff4f3439ab5f58e9768ff3da9" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40203 } } }
I20260807 00:06:04.011852 29715 sys_catalog.cc:458] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:04.012494 29722 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:04.015935 29722 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:04.022531 29722 catalog_manager.cc:1347] Generated new cluster ID: 0ad942398ff844768b4f112c3ee8386d
I20260807 00:06:04.022634 29722 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:04.031428 29722 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:04.032537 29722 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:04.042526 29722 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9: Generated new TSK 0
I20260807 00:06:04.043586 29722 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.30.204.65@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.65@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
Entry for principal kudu/127.30.204.65 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab.
I20260807 00:06:04.078208 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40203
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:04.201308 29745 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:04.201617 29745 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:04.201668 29745 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:04.205915 29745 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:04.205991 29745 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:04.206053 29745 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:06:04.212239 29745 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.65.keytab
--principal=kudu/127.30.204.65
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29745
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:04.213476 29745 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:04.214619 29745 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:04.225195 29757 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:04.225210 29756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:04.225216 29759 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:04.225442 29745 server_base.cc:1034] running on GCE node
I20260807 00:06:04.226027 29745 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:04.226761 29745 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:04.227957 29745 hybrid_clock.cc:648] HybridClock initialized: now 1786061164227922 us; error 51 us; skew 500 ppm
Aug 07 00:06:04 dist-test-slave-pww0 krb5kdc[29563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:04.231074 29745 init.cc:377] Logged in from keytab as kudu/127.30.204.65@KRBTEST.COM (short username kudu)
I20260807 00:06:04.232131 29745 webserver.cc:460] Webserver started at http://127.30.204.65:37687/ using document root <none> and password file <none>
I20260807 00:06:04.232749 29745 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:04.232828 29745 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:04.233050 29745 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:04.235272 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "ff1e91a5e60b45f6807bab827d3f73a5"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "539bfa3ea3b91dd4bcbfd0c4ce33de20"
server_key_iv: "514dd8a791f8777be5abc11074988729"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.235922 29745 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "ff1e91a5e60b45f6807bab827d3f73a5"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "539bfa3ea3b91dd4bcbfd0c4ce33de20"
server_key_iv: "514dd8a791f8777be5abc11074988729"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.240200 29745 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:04.242954 29769 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.244377 29745 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:04.244516 29745 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "ff1e91a5e60b45f6807bab827d3f73a5"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "539bfa3ea3b91dd4bcbfd0c4ce33de20"
server_key_iv: "514dd8a791f8777be5abc11074988729"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.244663 29745 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:04.255864 29745 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:04.259583 29745 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:04.259792 29745 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:04.260440 29745 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:04.261406 29745 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:04.261458 29745 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.261507 29745 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:04.261535 29745 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.270633 29745 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:35967
I20260807 00:06:04.270653 29966 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:35967 every 8 connection(s)
I20260807 00:06:04.271669 29745 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:06:04.276482 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 29745
I20260807 00:06:04.276636 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:06:04.276952 31537 external_mini_cluster.cc:1428] Setting key 79b1d014899337fe9695faeee419f40a
Aug 07 00:06:04 dist-test-slave-pww0 krb5kdc[29563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.65@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:04.287923 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.273741 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:56645 (local address 127.30.204.126:40203)
0807 00:06:04.273904 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.273908 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.274885 (+ 977us) server_negotiation.cc:407] Connection header received
0807 00:06:04.276373 (+ 1488us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.276377 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.276446 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.276525 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:04.278340 (+ 1815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.278972 (+ 632us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.280297 (+ 1325us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.280525 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.280628 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.283727 (+ 3099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:04.283730 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:04.283733 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:04.283763 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:04.285511 (+ 1748us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.286156 (+ 645us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.286160 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.286162 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.286269 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.286615 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.286619 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.286620 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.286882 (+ 262us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:04.286988 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.287575 (+ 587us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.287729 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":74}
WARNING: no policy specified for kudu/127.30.204.66@KRBTEST.COM; defaulting to no policy
I20260807 00:06:04.288829 29975 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.274143 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40203 (local address 127.30.204.65:56645)
0807 00:06:04.274668 (+ 525us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.274729 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.276114 (+ 1385us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.276768 (+ 654us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.276776 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.277405 (+ 629us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:04.278163 (+ 758us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.278181 (+ 18us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.279130 (+ 949us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.279133 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.280158 (+ 1025us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.280169 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.280690 (+ 521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.280694 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.280894 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.281598 (+ 704us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:04.281623 (+ 25us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:04.283500 (+ 1877us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:04.285681 (+ 2181us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:04.285688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:04.285721 (+ 33us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:04.286029 (+ 308us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:04.286360 (+ 331us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:04.286363 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:04.286364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:04.286518 (+ 154us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:04.286978 (+ 460us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:04.286984 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:04.287278 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.287443 (+ 165us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.287818 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":127,"threads_started":1}
I20260807 00:06:04.290225 29968 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40203
I20260807 00:06:04.290530 29968 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:04.291105 29968 heartbeater.cc:511] Master 127.30.204.126:40203 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.66@KRBTEST.COM" created.
I20260807 00:06:04.292865 29642 ts_manager.cc:194] Registered new tserver with Master: ff1e91a5e60b45f6807bab827d3f73a5 (127.30.204.65:35967)
I20260807 00:06:04.294075 29642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.65@KRBTEST.COM'} at 127.30.204.65:56645
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
Entry for principal kudu/127.30.204.66 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab.
I20260807 00:06:04.311102 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40203
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:04.435289 29979 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:04.435665 29979 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:04.435735 29979 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:04.439951 29979 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:04.440029 29979 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:04.440114 29979 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:06:04.445798 29979 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.66.keytab
--principal=kudu/127.30.204.66
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29979
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:04.447006 29979 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:04.448179 29979 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:04.458822 29988 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:04.458834 29994 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:04.459044 29987 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:04.459687 29979 server_base.cc:1034] running on GCE node
I20260807 00:06:04.460182 29979 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:04.460970 29979 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:04.462145 29979 hybrid_clock.cc:648] HybridClock initialized: now 1786061164462101 us; error 70 us; skew 500 ppm
Aug 07 00:06:04 dist-test-slave-pww0 krb5kdc[29563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:04.465509 29979 init.cc:377] Logged in from keytab as kudu/127.30.204.66@KRBTEST.COM (short username kudu)
I20260807 00:06:04.466672 29979 webserver.cc:460] Webserver started at http://127.30.204.66:46457/ using document root <none> and password file <none>
I20260807 00:06:04.467298 29979 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:04.467377 29979 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:04.467649 29979 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:04.469937 29979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "efe557dfdc1732a8537c8292db8d484a"
server_key_iv: "1046a549dd576fe5a663355378ec51a0"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.470563 29979 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "efe557dfdc1732a8537c8292db8d484a"
server_key_iv: "1046a549dd576fe5a663355378ec51a0"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.475101 29979 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 00:06:04.477978 30008 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.479549 29979 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:04.479682 29979 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "efe557dfdc1732a8537c8292db8d484a"
server_key_iv: "1046a549dd576fe5a663355378ec51a0"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.479821 29979 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:04.498525 29979 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:04.502061 29979 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:04.502246 29979 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:04.502876 29979 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:04.503808 29979 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:04.503860 29979 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.503918 29979 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:04.503942 29979 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.514086 29979 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:38153
I20260807 00:06:04.514140 30197 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:38153 every 8 connection(s)
I20260807 00:06:04.515074 29979 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:06:04.519752 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 29979
I20260807 00:06:04.519870 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:06:04.520180 31537 external_mini_cluster.cc:1428] Setting key c5cf7df5f63d18827956a8b8f1a76260
Aug 07 00:06:04 dist-test-slave-pww0 krb5kdc[29563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.66@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:04.532344 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.517496 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:34605 (local address 127.30.204.126:40203)
0807 00:06:04.517652 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.517657 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.518683 (+ 1026us) server_negotiation.cc:407] Connection header received
0807 00:06:04.520674 (+ 1991us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.520678 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.520744 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.520802 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:04.522865 (+ 2063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.523623 (+ 758us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.525536 (+ 1913us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.525798 (+ 262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.525905 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.528658 (+ 2753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:04.528661 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:04.528663 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:04.528693 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:04.530102 (+ 1409us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.530728 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.530731 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.530732 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.530815 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.531161 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.531165 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.531166 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.531362 (+ 196us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:04.531449 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.532034 (+ 585us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.532172 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:06:04.533439 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.517939 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40203 (local address 127.30.204.66:34605)
0807 00:06:04.518488 (+ 549us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.518553 (+ 65us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.520341 (+ 1788us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.521301 (+ 960us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.521313 (+ 12us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.521849 (+ 536us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:04.522659 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.522683 (+ 24us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.523791 (+ 1108us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.523796 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.525385 (+ 1589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.525402 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.525981 (+ 579us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.525986 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.526190 (+ 204us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.526787 (+ 597us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:04.526806 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:04.528458 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:04.530229 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:04.530236 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:04.530257 (+ 21us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:04.530626 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:04.530903 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:04.530906 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:04.530908 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:04.531063 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:04.531491 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:04.531500 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:04.531787 (+ 287us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.531972 (+ 185us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.532347 (+ 375us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":289,"thread_start_us":121,"threads_started":1}
I20260807 00:06:04.534689 30198 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40203
WARNING: no policy specified for kudu/127.30.204.67@KRBTEST.COM; defaulting to no policy
I20260807 00:06:04.534960 30198 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:04.535568 30198 heartbeater.cc:511] Master 127.30.204.126:40203 requested a full tablet report, sending...
I20260807 00:06:04.536761 29642 ts_manager.cc:194] Registered new tserver with Master: 26c70e3b6a2c427fbea0ce5dae53e2cc (127.30.204.66:38153)
I20260807 00:06:04.537369 29642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.66@KRBTEST.COM'} at 127.30.204.66:34605
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.67@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
Entry for principal kudu/127.30.204.67 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab.
I20260807 00:06:04.558393 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40203
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:45459
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:04.682606 30205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:04.682916 30205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:04.682962 30205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:04.687198 30205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:04.687278 30205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:04.687341 30205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:06:04.693171 30205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:45459
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.67.keytab
--principal=kudu/127.30.204.67
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:40203
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:04.694360 30205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:04.695422 30205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:04.705891 30216 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:04.705891 30213 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:04.705914 30212 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:04.706414 30205 server_base.cc:1034] running on GCE node
I20260807 00:06:04.706853 30205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:04.707556 30205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:04.708730 30205 hybrid_clock.cc:648] HybridClock initialized: now 1786061164708692 us; error 51 us; skew 500 ppm
Aug 07 00:06:04 dist-test-slave-pww0 krb5kdc[29563](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:04.711836 30205 init.cc:377] Logged in from keytab as kudu/127.30.204.67@KRBTEST.COM (short username kudu)
I20260807 00:06:04.712888 30205 webserver.cc:460] Webserver started at http://127.30.204.67:42373/ using document root <none> and password file <none>
I20260807 00:06:04.713502 30205 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:04.713570 30205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:04.713764 30205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:04.716037 30205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "e9e30088fa344ae58e17bf5e93a1b8ca"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "4bbde3d0fe877c0929f98f2fa9d9060a"
server_key_iv: "8c9bd340b571c7f5d7b0978a413c6fa1"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.716616 30205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "e9e30088fa344ae58e17bf5e93a1b8ca"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "4bbde3d0fe877c0929f98f2fa9d9060a"
server_key_iv: "8c9bd340b571c7f5d7b0978a413c6fa1"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.720674 30205 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:06:04.723333 30225 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.724761 30205 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:04.724888 30205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "e9e30088fa344ae58e17bf5e93a1b8ca"
format_stamp: "Formatted at 2026-08-07 00:06:04 on dist-test-slave-pww0"
server_key: "4bbde3d0fe877c0929f98f2fa9d9060a"
server_key_iv: "8c9bd340b571c7f5d7b0978a413c6fa1"
server_key_version: "encryptionkey@0"
I20260807 00:06:04.725020 30205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:04.737612 30205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:04.741079 30205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:04.741254 30205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:04.741859 30205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:04.742750 30205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:04.742810 30205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.742851 30205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:04.742872 30205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:04.752295 30205 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:42779
I20260807 00:06:04.752314 30398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:42779 every 8 connection(s)
I20260807 00:06:04.753265 30205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:06:04.757187 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 30205
I20260807 00:06:04.757323 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:06:04.757640 31537 external_mini_cluster.cc:1428] Setting key 6197c9fad4ad562303d3a50583f32c20
Aug 07 00:06:04 dist-test-slave-pww0 krb5kdc[29563](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061164, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.67@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:04.768472 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.755254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:45513 (local address 127.30.204.126:40203)
0807 00:06:04.755387 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.755390 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.756417 (+ 1027us) server_negotiation.cc:407] Connection header received
0807 00:06:04.758076 (+ 1659us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.758080 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.758149 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.758235 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:04.759779 (+ 1544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.760369 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.761657 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.761881 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.761957 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.764526 (+ 2569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:04.764530 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:04.764532 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:04.764561 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:04.766063 (+ 1502us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.766703 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.766706 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.766707 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.766810 (+ 103us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.767222 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.767227 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.767228 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.767530 (+ 302us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:04.767615 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.768205 (+ 590us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.768337 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:06:04.769433 30401 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.755678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:40203 (local address 127.30.204.67:45513)
0807 00:06:04.756243 (+ 565us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.756294 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.757803 (+ 1509us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.758470 (+ 667us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.758478 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.758994 (+ 516us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:04.759617 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.759634 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.760499 (+ 865us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.760501 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.761557 (+ 1056us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.761568 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.762018 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.762028 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.762190 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.762810 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:04.762827 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:04.764345 (+ 1518us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:04.766215 (+ 1870us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:04.766221 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:04.766238 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:04.766557 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:04.766905 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:04.766908 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:04.766909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:04.767081 (+ 172us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:04.767651 (+ 570us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:04.767657 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:04.767978 (+ 321us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.768150 (+ 172us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.768461 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":333,"thread_start_us":138,"threads_started":1}
I20260807 00:06:04.770860 30399 heartbeater.cc:347] Connected to a master server at 127.30.204.126:40203
I20260807 00:06:04.771142 30399 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:04.771859 30399 heartbeater.cc:511] Master 127.30.204.126:40203 requested a full tablet report, sending...
I20260807 00:06:04.773002 29642 ts_manager.cc:194] Registered new tserver with Master: e9e30088fa344ae58e17bf5e93a1b8ca (127.30.204.67:42779)
I20260807 00:06:04.773581 29642 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.30.204.67@KRBTEST.COM'} at 127.30.204.67:45513
I20260807 00:06:04.783900 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:06:04.792958 29711 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.786091 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49950 (local address 127.30.204.126:40203)
0807 00:06:04.786221 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.786225 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.786387 (+ 162us) server_negotiation.cc:407] Connection header received
0807 00:06:04.786499 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.786501 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.786544 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.786616 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:04.787279 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.787873 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.788706 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.788910 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.788979 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.790351 (+ 1372us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:04.790354 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:04.790356 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:04.790384 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:04.791564 (+ 1180us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.791945 (+ 381us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.791948 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.791949 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.792021 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:04.792207 (+ 186us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:04.792210 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:04.792211 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:04.792437 (+ 226us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:04.792541 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.792720 (+ 179us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.792825 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:06:04.796101 29642 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49950:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:06:04.799186 29642 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:06:04.815124 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.808873 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38008 (local address 127.30.204.67:42779)
0807 00:06:04.809756 (+ 883us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.809770 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.809794 (+ 24us) server_negotiation.cc:407] Connection header received
0807 00:06:04.809877 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.809883 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.810121 (+ 238us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.810309 (+ 188us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:04.810744 (+ 435us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.812060 (+ 1316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.813693 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.814323 (+ 630us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.814426 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.814593 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.814845 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.814960 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":758,"thread_start_us":174,"threads_started":1}
I20260807 00:06:04.815114 30421 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.808699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:42779 (local address 127.0.0.1:38008)
0807 00:06:04.809569 (+ 870us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.809583 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.809692 (+ 109us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.810397 (+ 705us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.810403 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.810425 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:04.810598 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.810605 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.812225 (+ 1620us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.812228 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.813515 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.813527 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.814451 (+ 924us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.814454 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.814566 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.814606 (+ 40us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.814800 (+ 194us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.814949 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":805,"mutex_wait_us":6,"thread_start_us":140,"threads_started":1}
I20260807 00:06:04.815140 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.809394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58618 (local address 127.30.204.65:35967)
0807 00:06:04.809977 (+ 583us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.809984 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.810275 (+ 291us) server_negotiation.cc:407] Connection header received
0807 00:06:04.810367 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.810372 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.810553 (+ 181us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.810722 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:04.810999 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.811919 (+ 920us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.813522 (+ 1603us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.814363 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.814474 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.814626 (+ 152us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.814915 (+ 289us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.815014 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":470,"thread_start_us":74,"threads_started":1}
I20260807 00:06:04.815621 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.808600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60278 (local address 127.30.204.66:38153)
0807 00:06:04.809418 (+ 818us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.809432 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.809837 (+ 405us) server_negotiation.cc:407] Connection header received
0807 00:06:04.809926 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.809931 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.810140 (+ 209us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.810309 (+ 169us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:04.810774 (+ 465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.812060 (+ 1286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.813767 (+ 1707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.814588 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.814691 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.814824 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.815356 (+ 532us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.815444 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":698,"thread_start_us":136,"threads_started":1}
I20260807 00:06:04.815814 30415 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.808406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:38153 (local address 127.0.0.1:60278)
0807 00:06:04.809493 (+ 1087us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.809554 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.809690 (+ 136us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.810497 (+ 807us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.810501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.810518 (+ 17us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:04.810670 (+ 152us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.810677 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.812218 (+ 1541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.812221 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.813633 (+ 1412us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.813643 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.814889 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.814892 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.814988 (+ 96us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.815010 (+ 22us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.815077 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.815152 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":896,"mutex_wait_us":242,"thread_start_us":97,"threads_started":1}
I20260807 00:06:04.816287 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.809209 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:35967 (local address 127.0.0.1:58618)
0807 00:06:04.810132 (+ 923us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.810145 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.810256 (+ 111us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.810795 (+ 539us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.810798 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.810813 (+ 15us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:04.810892 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.810897 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.812023 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.812027 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.813400 (+ 1373us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.813410 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.814674 (+ 1264us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.814677 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.814758 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.814782 (+ 24us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.816063 (+ 1281us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.816160 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":861,"mutex_wait_us":23,"thread_start_us":98,"threads_started":1}
I20260807 00:06:04.818679 30080 tablet_service.cc:1467] Processing CreateTablet for tablet f9732b56613f46d0862ae8ceda50b6e3 (DEFAULT_TABLE table=test-table [id=8e1804e0feb54df4af992bba99392690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:04.818810 30302 tablet_service.cc:1467] Processing CreateTablet for tablet f9732b56613f46d0862ae8ceda50b6e3 (DEFAULT_TABLE table=test-table [id=8e1804e0feb54df4af992bba99392690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:04.819072 29852 tablet_service.cc:1467] Processing CreateTablet for tablet f9732b56613f46d0862ae8ceda50b6e3 (DEFAULT_TABLE table=test-table [id=8e1804e0feb54df4af992bba99392690]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:04.819864 30080 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9732b56613f46d0862ae8ceda50b6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:04.819959 29852 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9732b56613f46d0862ae8ceda50b6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:04.819959 30302 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet f9732b56613f46d0862ae8ceda50b6e3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:04.826846 30429 tablet_bootstrap.cc:492] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Bootstrap starting.
I20260807 00:06:04.828128 30430 tablet_bootstrap.cc:492] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5: Bootstrap starting.
I20260807 00:06:04.828367 30434 tablet_bootstrap.cc:492] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Bootstrap starting.
I20260807 00:06:04.829872 30429 tablet_bootstrap.cc:654] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:04.830600 30434 tablet_bootstrap.cc:654] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:04.830600 30430 tablet_bootstrap.cc:654] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:04.831128 30429 log.cc:824] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:04.831517 30430 log.cc:824] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:04.831907 30434 log.cc:824] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:04.833182 30429 tablet_bootstrap.cc:492] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: No bootstrap required, opened a new log
I20260807 00:06:04.833402 30429 ts_tablet_manager.cc:1397] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Time spent bootstrapping tablet: real 0.007s user 0.004s sys 0.000s
I20260807 00:06:04.833783 30434 tablet_bootstrap.cc:492] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: No bootstrap required, opened a new log
I20260807 00:06:04.833971 30434 ts_tablet_manager.cc:1397] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:06:04.837007 30429 raft_consensus.cc:348] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.837275 30429 raft_consensus.cc:374] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:04.837344 30429 raft_consensus.cc:729] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26c70e3b6a2c427fbea0ce5dae53e2cc, State: Initialized, Role: FOLLOWER
I20260807 00:06:04.837617 30430 tablet_bootstrap.cc:492] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5: No bootstrap required, opened a new log
I20260807 00:06:04.837523 30434 raft_consensus.cc:348] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.837796 30434 raft_consensus.cc:374] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:04.837854 30434 raft_consensus.cc:729] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9e30088fa344ae58e17bf5e93a1b8ca, State: Initialized, Role: FOLLOWER
I20260807 00:06:04.837860 30430 ts_tablet_manager.cc:1397] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5: Time spent bootstrapping tablet: real 0.010s user 0.004s sys 0.000s
I20260807 00:06:04.837831 30429 consensus_queue.cc:260] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.838318 30434 consensus_queue.cc:260] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.838822 30198 heartbeater.cc:503] Master 127.30.204.126:40203 was elected leader, sending a full tablet report...
I20260807 00:06:04.838979 30429 ts_tablet_manager.cc:1428] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Time spent starting tablet: real 0.006s user 0.006s sys 0.000s
I20260807 00:06:04.839331 30434 ts_tablet_manager.cc:1428] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 00:06:04.839772 30399 heartbeater.cc:503] Master 127.30.204.126:40203 was elected leader, sending a full tablet report...
I20260807 00:06:04.842298 30430 raft_consensus.cc:348] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.842620 30430 raft_consensus.cc:374] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:04.842697 30430 raft_consensus.cc:729] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ff1e91a5e60b45f6807bab827d3f73a5, State: Initialized, Role: FOLLOWER
I20260807 00:06:04.843204 30430 consensus_queue.cc:260] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.843993 29968 heartbeater.cc:503] Master 127.30.204.126:40203 was elected leader, sending a full tablet report...
I20260807 00:06:04.844285 30430 ts_tablet_manager.cc:1428] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:06:04.886111 30439 raft_consensus.cc:482] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:06:04.886346 30439 raft_consensus.cc:504] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.887678 30439 leader_election.cc:288] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9e30088fa344ae58e17bf5e93a1b8ca (127.30.204.67:42779), ff1e91a5e60b45f6807bab827d3f73a5 (127.30.204.65:35967)
I20260807 00:06:04.892508 30423 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.888093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:58527 (local address 127.30.204.67:42779)
0807 00:06:04.888252 (+ 159us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.888257 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.888276 (+ 19us) server_negotiation.cc:407] Connection header received
0807 00:06:04.888378 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.888380 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.888429 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.888513 (+ 84us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:04.888793 (+ 280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.889569 (+ 776us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.891428 (+ 1859us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.891889 (+ 461us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.892003 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.892062 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.892291 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.892370 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260807 00:06:04.892508 30425 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.888101 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:35521 (local address 127.30.204.65:35967)
0807 00:06:04.888229 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.888233 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.888483 (+ 250us) server_negotiation.cc:407] Connection header received
0807 00:06:04.888567 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.888570 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.888622 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.888696 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:04.888942 (+ 246us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.889645 (+ 703us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.891390 (+ 1745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.891908 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.892031 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.892087 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.892296 (+ 209us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.892380 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:06:04.892508 30200 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.887927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:42779 (local address 127.30.204.66:58527)
0807 00:06:04.888147 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.888163 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.888268 (+ 105us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.888535 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.888538 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.888554 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:04.888670 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.888676 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.889734 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.889738 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.891257 (+ 1519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.891269 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.892026 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.892030 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.892155 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.892180 (+ 25us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.892251 (+ 71us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.892341 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"mutex_wait_us":158}
I20260807 00:06:04.892525 30442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.887970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:35967 (local address 127.30.204.66:35521)
0807 00:06:04.888394 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.888405 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:04.888486 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:04.888721 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:04.888723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:04.888741 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:04.888848 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.888853 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.889735 (+ 882us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.889738 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.891255 (+ 1517us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:04.891269 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.892047 (+ 778us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.892051 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:04.892159 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.892185 (+ 26us) client_negotiation.cc:769] Sending connection context
0807 00:06:04.892266 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:06:04.892344 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":365,"mutex_wait_us":40,"thread_start_us":182,"threads_started":1}
I20260807 00:06:04.893349 30339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9732b56613f46d0862ae8ceda50b6e3" candidate_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" is_pre_election: true
I20260807 00:06:04.893342 29889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9732b56613f46d0862ae8ceda50b6e3" candidate_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" is_pre_election: true
I20260807 00:06:04.893656 29889 raft_consensus.cc:2454] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c70e3b6a2c427fbea0ce5dae53e2cc in term 0.
I20260807 00:06:04.893659 30339 raft_consensus.cc:2454] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 26c70e3b6a2c427fbea0ce5dae53e2cc in term 0.
I20260807 00:06:04.894177 30009 leader_election.cc:302] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c70e3b6a2c427fbea0ce5dae53e2cc, ff1e91a5e60b45f6807bab827d3f73a5; no voters:
I20260807 00:06:04.894481 30439 raft_consensus.cc:2781] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:06:04.894578 30439 raft_consensus.cc:482] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:06:04.894626 30439 raft_consensus.cc:3037] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:04.895757 30439 raft_consensus.cc:504] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.896126 30439 leader_election.cc:288] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [CANDIDATE]: Term 1 election: Requested vote from peers e9e30088fa344ae58e17bf5e93a1b8ca (127.30.204.67:42779), ff1e91a5e60b45f6807bab827d3f73a5 (127.30.204.65:35967)
I20260807 00:06:04.896586 30339 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9732b56613f46d0862ae8ceda50b6e3" candidate_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca"
I20260807 00:06:04.896607 29889 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "f9732b56613f46d0862ae8ceda50b6e3" candidate_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ff1e91a5e60b45f6807bab827d3f73a5"
I20260807 00:06:04.896733 30339 raft_consensus.cc:3037] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:04.896750 29889 raft_consensus.cc:3037] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:04.897889 29889 raft_consensus.cc:2454] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c70e3b6a2c427fbea0ce5dae53e2cc in term 1.
I20260807 00:06:04.897898 30339 raft_consensus.cc:2454] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 26c70e3b6a2c427fbea0ce5dae53e2cc in term 1.
I20260807 00:06:04.898244 30009 leader_election.cc:302] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 26c70e3b6a2c427fbea0ce5dae53e2cc, ff1e91a5e60b45f6807bab827d3f73a5; no voters:
I20260807 00:06:04.898448 30439 raft_consensus.cc:2781] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:04.898697 30439 raft_consensus.cc:686] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 1 LEADER]: Becoming Leader. State: Replica: 26c70e3b6a2c427fbea0ce5dae53e2cc, State: Running, Role: LEADER
I20260807 00:06:04.899036 30439 consensus_queue.cc:237] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } }
I20260807 00:06:04.902709 29638 catalog_manager.cc:5495] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc reported cstate change: term changed from 0 to 1, leader changed from <none> to 26c70e3b6a2c427fbea0ce5dae53e2cc (127.30.204.66). New cstate: current_term: 1 leader_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "26c70e3b6a2c427fbea0ce5dae53e2cc" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 38153 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 } health_report { overall_health: UNKNOWN } } }
I20260807 00:06:04.920440 30418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.916315 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60294 (local address 127.30.204.66:38153)
0807 00:06:04.916464 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:04.916467 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:04.916483 (+ 16us) server_negotiation.cc:407] Connection header received
0807 00:06:04.916671 (+ 188us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:04.916674 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:04.916767 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:04.916872 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:04.917187 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.917740 (+ 553us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.918666 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:04.918892 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:04.918980 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:04.919412 (+ 432us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:04.919966 (+ 554us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:04.920087 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:04.920178 (+ 91us) server_negotiation.cc:299] Negotiation successful
0807 00:06:04.920245 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 00:06:04.927600 30339 raft_consensus.cc:1261] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 1 FOLLOWER]: Refusing update from remote peer 26c70e3b6a2c427fbea0ce5dae53e2cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:04.927600 29889 raft_consensus.cc:1261] T f9732b56613f46d0862ae8ceda50b6e3 P ff1e91a5e60b45f6807bab827d3f73a5 [term 1 FOLLOWER]: Refusing update from remote peer 26c70e3b6a2c427fbea0ce5dae53e2cc: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:04.928493 30439 consensus_queue.cc:1035] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9e30088fa344ae58e17bf5e93a1b8ca" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 42779 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:04.928730 30443 consensus_queue.cc:1035] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [LEADER]: Connected to new peer: Peer: permanent_uuid: "ff1e91a5e60b45f6807bab827d3f73a5" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 35967 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:04.938184 30453 mvcc.cc:205] Tried to move back new op lower bound from 7315706531534032896 to 7315706531429367808. Current Snapshot: MvccSnapshot[applied={T|T < 7315706531534032896}]
I20260807 00:06:04.947975 29638 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:04.948202 29638 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49950:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:04.951066 29638 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e91a7ffff4f3439ab5f58e9768ff3da9: Sending DeleteTablet for 3 replicas of tablet f9732b56613f46d0862ae8ceda50b6e3
I20260807 00:06:04.952071 30080 tablet_service.cc:1514] Processing DeleteTablet for tablet f9732b56613f46d0862ae8ceda50b6e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:04 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:60278
I20260807 00:06:04.952071 29852 tablet_service.cc:1514] Processing DeleteTablet for tablet f9732b56613f46d0862ae8ceda50b6e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:04 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:58618
I20260807 00:06:04.952067 30302 tablet_service.cc:1514] Processing DeleteTablet for tablet f9732b56613f46d0862ae8ceda50b6e3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:04 UTC) from {username='kudu', principal='kudu/127.30.204.126@KRBTEST.COM'} at 127.0.0.1:38008
I20260807 00:06:04.952703 30470 tablet_replica.cc:330] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: stopping tablet replica
I20260807 00:06:04.952888 30473 tablet_replica.cc:330] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: stopping tablet replica
I20260807 00:06:04.953083 30470 raft_consensus.cc:2229] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:06:04.953204 30473 raft_consensus.cc:2229] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:06:04.953572 30473 raft_consensus.cc:2258] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:04.953543 30470 raft_consensus.cc:2258] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:04.953702 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 29745
I20260807 00:06:04.961025 30473 ts_tablet_manager.cc:1905] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:04.963021 30470 ts_tablet_manager.cc:1905] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:04.964455 30473 ts_tablet_manager.cc:1918] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:04.964593 30473 log.cc:1196] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/f9732b56613f46d0862ae8ceda50b6e3
I20260807 00:06:04.964957 30473 ts_tablet_manager.cc:1939] T f9732b56613f46d0862ae8ceda50b6e3 P e9e30088fa344ae58e17bf5e93a1b8ca: Deleting consensus metadata
I20260807 00:06:04.966409 30470 ts_tablet_manager.cc:1918] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:04.966616 30470 log.cc:1196] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/f9732b56613f46d0862ae8ceda50b6e3
I20260807 00:06:04.966544 29621 catalog_manager.cc:4841] TS e9e30088fa344ae58e17bf5e93a1b8ca (127.30.204.67:42779): tablet f9732b56613f46d0862ae8ceda50b6e3 (table test-table [id=8e1804e0feb54df4af992bba99392690]) successfully deleted
I20260807 00:06:04.967017 30470 ts_tablet_manager.cc:1939] T f9732b56613f46d0862ae8ceda50b6e3 P 26c70e3b6a2c427fbea0ce5dae53e2cc: Deleting consensus metadata
W20260807 00:06:04.967083 29613 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:35967 (error 108)
I20260807 00:06:04.968263 30424 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:04.967954 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:35967 (local address 127.0.0.1:58622)
0807 00:06:04.968086 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:04.968165 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:35967: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":48}
I20260807 00:06:04.968379 29613 catalog_manager.cc:4841] TS 26c70e3b6a2c427fbea0ce5dae53e2cc (127.30.204.66:38153): tablet f9732b56613f46d0862ae8ceda50b6e3 (table test-table [id=8e1804e0feb54df4af992bba99392690]) successfully deleted
W20260807 00:06:04.968586 29613 catalog_manager.cc:4584] TS ff1e91a5e60b45f6807bab827d3f73a5 (127.30.204.65:35967): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet f9732b56613f46d0862ae8ceda50b6e3: Network error: Client connection negotiation failed: client connection to 127.30.204.65:35967: connect: Connection refused (error 111)
I20260807 00:06:04.968998 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 29979
I20260807 00:06:04.983387 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 30205
I20260807 00:06:05.001009 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 29584
2026-08-07T00:06:05Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2515 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-08-07T00:06:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:05Z Disabled control of system clock
I20260807 00:06:05.047194 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39819
--webserver_interface=127.30.204.126
--webserver_port=0
--builtin_ntp_servers=127.30.204.84:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:39819
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:06:05.166704 30479 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:05.167049 30479 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:05.167091 30479 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:05.171393 30479 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:05.171514 30479 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:05.171530 30479 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:05.171545 30479 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:05.171559 30479 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:05.176748 30479 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:39819
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39819
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30479
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:05.177975 30479 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:05.179562 30479 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:05.188763 30487 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.188750 30493 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.188750 30488 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:05.188977 30479 server_base.cc:1034] running on GCE node
I20260807 00:06:05.189692 30479 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:05.190940 30479 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:05.192109 30479 hybrid_clock.cc:648] HybridClock initialized: now 1786061165192091 us; error 31 us; skew 500 ppm
I20260807 00:06:05.194231 30479 webserver.cc:460] Webserver started at http://127.30.204.126:36029/ using document root <none> and password file <none>
I20260807 00:06:05.194861 30479 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:05.194936 30479 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:05.195145 30479 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:05.197477 30479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "69f259c223c74dc8a50ad7deb08830d7"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "cf2201a8cc61fb35d66a4ded95916be6"
server_key_iv: "64d80ee8b0128ee49eac7c7bbb269b84"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.198113 30479 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "69f259c223c74dc8a50ad7deb08830d7"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "cf2201a8cc61fb35d66a4ded95916be6"
server_key_iv: "64d80ee8b0128ee49eac7c7bbb269b84"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.202562 30479 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:06:05.205341 30500 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.206729 30479 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:05.206874 30479 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "69f259c223c74dc8a50ad7deb08830d7"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "cf2201a8cc61fb35d66a4ded95916be6"
server_key_iv: "64d80ee8b0128ee49eac7c7bbb269b84"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.207000 30479 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:05.227226 30479 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:05.228071 30479 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:05.228291 30479 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:05.235594 30479 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:39819
I20260807 00:06:05.235628 30591 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:39819 every 8 connection(s)
I20260807 00:06:05.236655 30479 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:05.240051 30594 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:05.244450 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 30479
I20260807 00:06:05.244592 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:05.244879 31537 external_mini_cluster.cc:1428] Setting key e5082b82e64bd11ffc4067c7bfbb41cc
I20260807 00:06:05.246476 30594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7: Bootstrap starting.
I20260807 00:06:05.249487 30594 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:05.250501 30594 log.cc:824] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:05.252785 30594 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7: No bootstrap required, opened a new log
I20260807 00:06:05.255618 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.246429 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47716 (local address 127.30.204.126:39819)
0807 00:06:05.247210 (+ 781us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.247242 (+ 32us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.247287 (+ 45us) server_negotiation.cc:407] Connection header received
0807 00:06:05.248204 (+ 917us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.248242 (+ 38us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.248784 (+ 542us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.249173 (+ 389us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:05.249707 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.251176 (+ 1469us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.252211 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.252732 (+ 521us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.252889 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.253767 (+ 878us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:05.253773 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:05.253788 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:05.254198 (+ 410us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:05.254267 (+ 69us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:05.254288 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:05.254375 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.254518 (+ 143us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.254802 (+ 284us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":173,"threads_started":1}
I20260807 00:06:05.256955 30594 raft_consensus.cc:348] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f259c223c74dc8a50ad7deb08830d7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39819 } }
I20260807 00:06:05.257261 30594 raft_consensus.cc:374] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:05.257323 30594 raft_consensus.cc:729] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 69f259c223c74dc8a50ad7deb08830d7, State: Initialized, Role: FOLLOWER
I20260807 00:06:05.257805 30594 consensus_queue.cc:260] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f259c223c74dc8a50ad7deb08830d7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39819 } }
I20260807 00:06:05.257979 30594 raft_consensus.cc:388] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:05.258049 30594 raft_consensus.cc:482] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:05.258154 30594 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:05.259159 30594 raft_consensus.cc:504] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f259c223c74dc8a50ad7deb08830d7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39819 } }
I20260807 00:06:05.259543 30594 leader_election.cc:302] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 69f259c223c74dc8a50ad7deb08830d7; no voters:
I20260807 00:06:05.259879 30594 leader_election.cc:288] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:05.259996 30602 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:05.260232 30602 raft_consensus.cc:686] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [term 1 LEADER]: Becoming Leader. State: Replica: 69f259c223c74dc8a50ad7deb08830d7, State: Running, Role: LEADER
I20260807 00:06:05.260533 30602 consensus_queue.cc:237] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f259c223c74dc8a50ad7deb08830d7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39819 } }
I20260807 00:06:05.261148 30594 sys_catalog.cc:564] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:05.262269 30603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "69f259c223c74dc8a50ad7deb08830d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f259c223c74dc8a50ad7deb08830d7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39819 } } }
I20260807 00:06:05.262415 30603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:05.262708 30606 sys_catalog.cc:455] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 69f259c223c74dc8a50ad7deb08830d7. Latest consensus state: current_term: 1 leader_uuid: "69f259c223c74dc8a50ad7deb08830d7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "69f259c223c74dc8a50ad7deb08830d7" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39819 } } }
I20260807 00:06:05.262807 30617 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:05.262823 30606 sys_catalog.cc:458] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:05.266069 30617 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:05.273300 30617 catalog_manager.cc:1347] Generated new cluster ID: b4f3019f33864efca7b7708b87294630
I20260807 00:06:05.273452 30617 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:05.285197 30617 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:05.286628 30617 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:05.295121 30617 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7: Generated new TSK 0
I20260807 00:06:05.296034 30617 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:05.300276 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.65:0
--local_ip_for_outbound_sockets=127.30.204.65
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39819
--builtin_ntp_servers=127.30.204.84:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260807 00:06:05.424564 30630 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:05.424880 30630 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:05.424918 30630 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:05.429247 30630 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:05.429325 30630 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:05.429384 30630 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.65
I20260807 00:06:05.434520 30630 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.65:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.30.204.65
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30630
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.65
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:05.435813 30630 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:05.437338 30630 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:06:05.448027 30630 server_base.cc:1034] running on GCE node
W20260807 00:06:05.447983 30640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.447983 30639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.447985 30642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:05.448764 30630 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:05.449447 30630 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:05.450629 30630 hybrid_clock.cc:648] HybridClock initialized: now 1786061165450596 us; error 46 us; skew 500 ppm
I20260807 00:06:05.452747 30630 webserver.cc:460] Webserver started at http://127.30.204.65:44659/ using document root <none> and password file <none>
I20260807 00:06:05.453344 30630 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:05.453410 30630 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:05.453589 30630 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:05.455943 30630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data/instance:
uuid: "5f6e5f1ae4334139a127b1816a2a4b46"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "1a5c62aec494eb57ecaea8ae28ac3634"
server_key_iv: "7599f9b92997c374d1ea9ecb573ed70a"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.456534 30630 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance:
uuid: "5f6e5f1ae4334139a127b1816a2a4b46"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "1a5c62aec494eb57ecaea8ae28ac3634"
server_key_iv: "7599f9b92997c374d1ea9ecb573ed70a"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.460659 30630 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:06:05.463704 30653 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.465159 30630 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:05.465330 30630 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/wal
uuid: "5f6e5f1ae4334139a127b1816a2a4b46"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "1a5c62aec494eb57ecaea8ae28ac3634"
server_key_iv: "7599f9b92997c374d1ea9ecb573ed70a"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.465464 30630 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:05.492712 30630 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:05.493510 30630 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:05.493754 30630 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:05.494416 30630 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:05.495513 30630 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:05.495594 30630 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.495666 30630 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:05.495694 30630 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.505277 30630 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.65:46389
I20260807 00:06:05.505358 30838 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.65:46389 every 8 connection(s)
I20260807 00:06:05.506377 30630 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/data/info.pb
I20260807 00:06:05.508777 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 30630
I20260807 00:06:05.508908 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-0/wal/instance
I20260807 00:06:05.509263 31537 external_mini_cluster.cc:1428] Setting key 30764884eebec17dc684828402861c1e
I20260807 00:06:05.511425 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.66:0
--local_ip_for_outbound_sockets=127.30.204.66
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39819
--builtin_ntp_servers=127.30.204.84:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:06:05.518006 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.508402 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.65:57225 (local address 127.30.204.126:39819)
0807 00:06:05.508558 (+ 156us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.508562 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.510146 (+ 1584us) server_negotiation.cc:407] Connection header received
0807 00:06:05.511893 (+ 1747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.511897 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.511959 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.512045 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:05.513136 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.513759 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.515533 (+ 1774us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.515808 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.515905 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.517419 (+ 1514us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:05.517423 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:05.517425 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:05.517441 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:05.517465 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:05.517473 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:05.517543 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.517795 (+ 252us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.517865 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:06:05.519338 30842 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.508949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39819 (local address 127.30.204.65:57225)
0807 00:06:05.509962 (+ 1013us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:05.510030 (+ 68us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:05.511615 (+ 1585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:05.512232 (+ 617us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:05.512240 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:05.512344 (+ 104us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:05.512947 (+ 603us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.512963 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.513933 (+ 970us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.513938 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.515328 (+ 1390us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.515342 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.515996 (+ 654us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.516000 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.516225 (+ 225us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.517023 (+ 798us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:06:05.517025 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:05.517093 (+ 68us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:06:05.517212 (+ 119us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:05.517566 (+ 354us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:05.517576 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:05.517582 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:06:05.517745 (+ 163us) client_negotiation.cc:240] Negotiation successful
0807 00:06:05.518131 (+ 386us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"thread_start_us":161,"threads_started":1}
I20260807 00:06:05.520874 30840 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39819
I20260807 00:06:05.521129 30840 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:05.521730 30840 heartbeater.cc:511] Master 127.30.204.126:39819 requested a full tablet report, sending...
I20260807 00:06:05.523630 30529 ts_manager.cc:194] Registered new tserver with Master: 5f6e5f1ae4334139a127b1816a2a4b46 (127.30.204.65:46389)
I20260807 00:06:05.525357 30529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.65:57225
W20260807 00:06:05.634680 30843 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:05.634990 30843 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:05.635030 30843 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:05.639149 30843 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:05.639248 30843 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:05.639310 30843 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.66
I20260807 00:06:05.644557 30843 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.66:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.30.204.66
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30843
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.66
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:05.645695 30843 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:05.647133 30843 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:05.657827 30849 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.658306 30853 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.658311 30851 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:05.658674 30843 server_base.cc:1034] running on GCE node
I20260807 00:06:05.659194 30843 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:05.660006 30843 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:05.661219 30843 hybrid_clock.cc:648] HybridClock initialized: now 1786061165661196 us; error 37 us; skew 500 ppm
I20260807 00:06:05.663417 30843 webserver.cc:460] Webserver started at http://127.30.204.66:43453/ using document root <none> and password file <none>
I20260807 00:06:05.664074 30843 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:05.664155 30843 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:05.664366 30843 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:05.666690 30843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data/instance:
uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "3f54430a47d70180d9ced46e6aa1b9a2"
server_key_iv: "39291bea8dfd3ec81c07b30468d86c17"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.667299 30843 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance:
uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "3f54430a47d70180d9ced46e6aa1b9a2"
server_key_iv: "39291bea8dfd3ec81c07b30468d86c17"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.671784 30843 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:06:05.674652 30859 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.676213 30843 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:05.676354 30843 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal
uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "3f54430a47d70180d9ced46e6aa1b9a2"
server_key_iv: "39291bea8dfd3ec81c07b30468d86c17"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.676477 30843 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:05.697407 30843 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:05.698174 30843 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:05.698374 30843 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:05.699038 30843 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:05.700100 30843 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:05.700155 30843 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.700217 30843 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:05.700243 30843 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.709293 30843 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.66:40331
I20260807 00:06:05.709329 31056 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.66:40331 every 8 connection(s)
I20260807 00:06:05.710295 30843 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/data/info.pb
I20260807 00:06:05.710562 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 30843
I20260807 00:06:05.710686 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal/instance
I20260807 00:06:05.711052 31537 external_mini_cluster.cc:1428] Setting key 157e69206dfd2baaf3e4fe44408b9388
I20260807 00:06:05.713341 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.30.204.67:0
--local_ip_for_outbound_sockets=127.30.204.67
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39819
--builtin_ntp_servers=127.30.204.84:33705
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260807 00:06:05.721458 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.712387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:47855 (local address 127.30.204.126:39819)
0807 00:06:05.712525 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.712529 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.713930 (+ 1401us) server_negotiation.cc:407] Connection header received
0807 00:06:05.715544 (+ 1614us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.715548 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.715604 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.715685 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:05.716934 (+ 1249us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.717485 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.719268 (+ 1783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.719547 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.719633 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.720970 (+ 1337us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:05.720973 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:05.720976 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:05.720991 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:05.721011 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:05.721019 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:05.721074 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.721260 (+ 186us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.721326 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260807 00:06:05.722766 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.712952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39819 (local address 127.30.204.66:47855)
0807 00:06:05.713780 (+ 828us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:05.713825 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:05.715212 (+ 1387us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:05.715907 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:05.715917 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:05.716037 (+ 120us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:05.716770 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.716791 (+ 21us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.717626 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.717629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.719112 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.719126 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.719717 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.719722 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.719941 (+ 219us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.720623 (+ 682us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:06:05.720625 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:05.720691 (+ 66us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:06:05.720794 (+ 103us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:05.721097 (+ 303us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:05.721107 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:05.721113 (+ 6us) client_negotiation.cc:769] Sending connection context
0807 00:06:05.721261 (+ 148us) client_negotiation.cc:240] Negotiation successful
0807 00:06:05.721604 (+ 343us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":346,"thread_start_us":158,"threads_started":1}
I20260807 00:06:05.724130 31057 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39819
I20260807 00:06:05.724435 31057 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:05.725167 31057 heartbeater.cc:511] Master 127.30.204.126:39819 requested a full tablet report, sending...
I20260807 00:06:05.726415 30529 ts_manager.cc:194] Registered new tserver with Master: dfe39b4f9f9f4da49cc5bf6eae15df5c (127.30.204.66:40331)
I20260807 00:06:05.727010 30529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.66:47855
W20260807 00:06:05.837772 31061 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:05.838083 31061 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:05.838140 31061 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:05.842415 31061 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:05.842489 31061 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:05.842569 31061 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.30.204.67
I20260807 00:06:05.847675 31061 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:33705
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.30.204.67:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.30.204.67
--webserver_port=0
--tserver_master_addrs=127.30.204.126:39819
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31061
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.30.204.67
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:05.848886 31061 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:05.850396 31061 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:06:05.861142 31061 server_base.cc:1034] running on GCE node
W20260807 00:06:05.861050 31068 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.861050 31071 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:05.861060 31067 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:05.861963 31061 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:05.862753 31061 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:05.863924 31061 hybrid_clock.cc:648] HybridClock initialized: now 1786061165863883 us; error 57 us; skew 500 ppm
I20260807 00:06:05.865949 31061 webserver.cc:460] Webserver started at http://127.30.204.67:33683/ using document root <none> and password file <none>
I20260807 00:06:05.866566 31061 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:05.866641 31061 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:05.866883 31061 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:05.869289 31061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data/instance:
uuid: "7c327a056abd434c8a5696dd5abd8e7b"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "ecc4788d8171c11f6011f6d26b634f5a"
server_key_iv: "1b5fb9b855047c8c55d5499ad24f5f10"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.869975 31061 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance:
uuid: "7c327a056abd434c8a5696dd5abd8e7b"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "ecc4788d8171c11f6011f6d26b634f5a"
server_key_iv: "1b5fb9b855047c8c55d5499ad24f5f10"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.874229 31061 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.002s
I20260807 00:06:05.877416 31082 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.878976 31061 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:05.879142 31061 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal
uuid: "7c327a056abd434c8a5696dd5abd8e7b"
format_stamp: "Formatted at 2026-08-07 00:06:05 on dist-test-slave-pww0"
server_key: "ecc4788d8171c11f6011f6d26b634f5a"
server_key_iv: "1b5fb9b855047c8c55d5499ad24f5f10"
server_key_version: "encryptionkey@0"
I20260807 00:06:05.879271 31061 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:05.909576 31061 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:05.910398 31061 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:05.910610 31061 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:05.911330 31061 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260807 00:06:05.912420 31061 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260807 00:06:05.912472 31061 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.912531 31061 ts_tablet_manager.cc:610] Registered 0 tablets
I20260807 00:06:05.912563 31061 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:05.921561 31061 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.67:38641
I20260807 00:06:05.921588 31280 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.67:38641 every 8 connection(s)
I20260807 00:06:05.922763 31061 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/data/info.pb
I20260807 00:06:05.932394 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31061
I20260807 00:06:05.932509 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal/instance
I20260807 00:06:05.932824 31537 external_mini_cluster.cc:1428] Setting key c6ee52a7ab5beb354a3bdcf841496570
I20260807 00:06:05.933072 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.924979 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.67:43481 (local address 127.30.204.126:39819)
0807 00:06:05.925134 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.925138 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.926346 (+ 1208us) server_negotiation.cc:407] Connection header received
0807 00:06:05.927887 (+ 1541us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.927891 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.927957 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.928027 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:05.929040 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.929576 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.930789 (+ 1213us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.931013 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.931105 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.932374 (+ 1269us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:05.932378 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:05.932380 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:05.932397 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:05.932432 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:05.932443 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:05.932586 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.932830 (+ 244us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.932910 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260807 00:06:05.934082 31286 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.925422 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:39819 (local address 127.30.204.67:43481)
0807 00:06:05.926172 (+ 750us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:05.926219 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:05.927608 (+ 1389us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:05.928238 (+ 630us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:05.928245 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:05.928340 (+ 95us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:05.928879 (+ 539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.928893 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.929713 (+ 820us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.929716 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.930659 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.930670 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.931175 (+ 505us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.931179 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.931350 (+ 171us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.932019 (+ 669us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0807 00:06:05.932021 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:05.932066 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0807 00:06:05.932160 (+ 94us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:05.932597 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:05.932608 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:05.932615 (+ 7us) client_negotiation.cc:769] Sending connection context
0807 00:06:05.932784 (+ 169us) client_negotiation.cc:240] Negotiation successful
0807 00:06:05.933113 (+ 329us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":321,"thread_start_us":137,"threads_started":1}
I20260807 00:06:05.935333 31282 heartbeater.cc:347] Connected to a master server at 127.30.204.126:39819
I20260807 00:06:05.935618 31282 heartbeater.cc:464] Registering TS with master...
I20260807 00:06:05.936199 31282 heartbeater.cc:511] Master 127.30.204.126:39819 requested a full tablet report, sending...
I20260807 00:06:05.937335 30529 ts_manager.cc:194] Registered new tserver with Master: 7c327a056abd434c8a5696dd5abd8e7b (127.30.204.67:38641)
I20260807 00:06:05.937916 30529 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.30.204.67:43481
I20260807 00:06:05.947507 31537 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260807 00:06:05.953456 30598 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.949664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47724 (local address 127.30.204.126:39819)
0807 00:06:05.949849 (+ 185us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.949854 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.949957 (+ 103us) server_negotiation.cc:407] Connection header received
0807 00:06:05.950092 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.950095 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.950141 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.950207 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:05.950516 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.951043 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.951944 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.952164 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.952257 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.953020 (+ 763us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:05.953024 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:05.953026 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:05.953040 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:05.953059 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:05.953068 (+ 9us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:05.953125 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.953238 (+ 113us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.953300 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260807 00:06:05.956789 30529 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:47724:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260807 00:06:05.960048 30529 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260807 00:06:05.976671 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.970759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.66:40331 (local address 127.0.0.1:54396)
0807 00:06:05.971499 (+ 740us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:05.971513 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:05.971603 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:05.972136 (+ 533us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:05.972139 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:05.972165 (+ 26us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:05.972266 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.972272 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.973286 (+ 1014us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.973290 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.974752 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.974762 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.975954 (+ 1192us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.975958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.976144 (+ 186us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.976172 (+ 28us) client_negotiation.cc:769] Sending connection context
0807 00:06:05.976259 (+ 87us) client_negotiation.cc:240] Negotiation successful
0807 00:06:05.976484 (+ 225us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":661,"mutex_wait_us":182,"thread_start_us":127,"threads_started":1}
I20260807 00:06:05.976717 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.971301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54396 (local address 127.30.204.66:40331)
0807 00:06:05.971685 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.971698 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.971723 (+ 25us) server_negotiation.cc:407] Connection header received
0807 00:06:05.971799 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.971804 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.971995 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.972154 (+ 159us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:05.972361 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.973171 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.974885 (+ 1714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.975799 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.976129 (+ 330us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.976353 (+ 224us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.976413 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.976526 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":149,"threads_started":1}
I20260807 00:06:05.976918 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.969965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43986 (local address 127.30.204.67:38641)
0807 00:06:05.971340 (+ 1375us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.971354 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.971382 (+ 28us) server_negotiation.cc:407] Connection header received
0807 00:06:05.971492 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.971498 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.971724 (+ 226us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.971886 (+ 162us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:05.972208 (+ 322us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.973346 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.974944 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.975799 (+ 855us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.976415 (+ 616us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.976601 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.976652 (+ 51us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.976760 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1253,"thread_start_us":201,"threads_started":1}
I20260807 00:06:05.977393 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.970742 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53966 (local address 127.30.204.65:46389)
0807 00:06:05.971845 (+ 1103us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:05.971851 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:05.971871 (+ 20us) server_negotiation.cc:407] Connection header received
0807 00:06:05.971936 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:05.971940 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:05.972086 (+ 146us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:05.972225 (+ 139us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:05.972615 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.973723 (+ 1108us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.975094 (+ 1371us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.975749 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.975857 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.976102 (+ 245us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:05.976489 (+ 387us) server_negotiation.cc:299] Negotiation successful
0807 00:06:05.976602 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":992,"thread_start_us":112,"threads_started":1}
I20260807 00:06:05.977432 31303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.970018 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:46389 (local address 127.0.0.1:53966)
0807 00:06:05.970812 (+ 794us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:05.970876 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:05.971028 (+ 152us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:05.972373 (+ 1345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:05.972376 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:05.972394 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:05.972483 (+ 89us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.972488 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.973852 (+ 1364us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.973856 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.974980 (+ 1124us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.974987 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.975880 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.975883 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.976043 (+ 160us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.976337 (+ 294us) client_negotiation.cc:769] Sending connection context
0807 00:06:05.976383 (+ 46us) client_negotiation.cc:240] Negotiation successful
0807 00:06:05.977282 (+ 899us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":693,"mutex_wait_us":587,"spinlock_wait_cycles":2304,"thread_start_us":145,"threads_started":1}
I20260807 00:06:05.978402 31299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:05.969790 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:38641 (local address 127.0.0.1:43986)
0807 00:06:05.970813 (+ 1023us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:05.970876 (+ 63us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:05.971027 (+ 151us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:05.971891 (+ 864us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:05.971898 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:05.971921 (+ 23us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:05.972067 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.972074 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.973481 (+ 1407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.973485 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.974828 (+ 1343us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:05.974840 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:05.975954 (+ 1114us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:05.975958 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:05.976111 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:05.976161 (+ 50us) client_negotiation.cc:769] Sending connection context
0807 00:06:05.976269 (+ 108us) client_negotiation.cc:240] Negotiation successful
0807 00:06:05.978257 (+ 1988us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":822,"mutex_wait_us":1831,"spinlock_wait_cycles":12928,"thread_start_us":95,"threads_started":1}
I20260807 00:06:05.980741 30944 tablet_service.cc:1467] Processing CreateTablet for tablet 43fea0acd275413e9936b251874457a1 (DEFAULT_TABLE table=test-table [id=44dfd4e6f2154abe8376f11e4a27bf2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:05.981148 31169 tablet_service.cc:1467] Processing CreateTablet for tablet 43fea0acd275413e9936b251874457a1 (DEFAULT_TABLE table=test-table [id=44dfd4e6f2154abe8376f11e4a27bf2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:05.981164 30728 tablet_service.cc:1467] Processing CreateTablet for tablet 43fea0acd275413e9936b251874457a1 (DEFAULT_TABLE table=test-table [id=44dfd4e6f2154abe8376f11e4a27bf2a]), partition=RANGE (key) PARTITION UNBOUNDED
I20260807 00:06:05.982117 31169 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43fea0acd275413e9936b251874457a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:05.982120 30944 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43fea0acd275413e9936b251874457a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:05.982163 30728 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 43fea0acd275413e9936b251874457a1. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:05.990062 31308 tablet_bootstrap.cc:492] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Bootstrap starting.
I20260807 00:06:05.991487 31310 tablet_bootstrap.cc:492] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: Bootstrap starting.
I20260807 00:06:05.991814 31312 tablet_bootstrap.cc:492] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Bootstrap starting.
I20260807 00:06:05.993161 31308 tablet_bootstrap.cc:654] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:05.993908 31312 tablet_bootstrap.cc:654] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:05.993908 31310 tablet_bootstrap.cc:654] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:05.994024 31308 log.cc:824] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:05.994799 31310 log.cc:824] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:05.994799 31312 log.cc:824] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:05.996428 31308 tablet_bootstrap.cc:492] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: No bootstrap required, opened a new log
I20260807 00:06:05.996672 31308 ts_tablet_manager.cc:1397] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:06:05.996902 31310 tablet_bootstrap.cc:492] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: No bootstrap required, opened a new log
I20260807 00:06:05.997098 31310 ts_tablet_manager.cc:1397] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: Time spent bootstrapping tablet: real 0.006s user 0.004s sys 0.000s
I20260807 00:06:05.997264 31312 tablet_bootstrap.cc:492] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: No bootstrap required, opened a new log
I20260807 00:06:05.998764 31312 ts_tablet_manager.cc:1397] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Time spent bootstrapping tablet: real 0.007s user 0.005s sys 0.000s
I20260807 00:06:06.000504 31310 raft_consensus.cc:348] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.000778 31310 raft_consensus.cc:374] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:06.000835 31310 raft_consensus.cc:729] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5f6e5f1ae4334139a127b1816a2a4b46, State: Initialized, Role: FOLLOWER
I20260807 00:06:06.001299 31310 consensus_queue.cc:260] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.001187 31308 raft_consensus.cc:348] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.001459 31308 raft_consensus.cc:374] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:06.001531 31308 raft_consensus.cc:729] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7c327a056abd434c8a5696dd5abd8e7b, State: Initialized, Role: FOLLOWER
I20260807 00:06:06.002038 30840 heartbeater.cc:503] Master 127.30.204.126:39819 was elected leader, sending a full tablet report...
I20260807 00:06:06.002063 31308 consensus_queue.cc:260] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.002442 31310 ts_tablet_manager.cc:1428] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: Time spent starting tablet: real 0.005s user 0.006s sys 0.000s
I20260807 00:06:06.003130 31308 ts_tablet_manager.cc:1428] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Time spent starting tablet: real 0.006s user 0.007s sys 0.000s
I20260807 00:06:06.003279 31312 raft_consensus.cc:348] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.003564 31282 heartbeater.cc:503] Master 127.30.204.126:39819 was elected leader, sending a full tablet report...
I20260807 00:06:06.003580 31312 raft_consensus.cc:374] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:06.003649 31312 raft_consensus.cc:729] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dfe39b4f9f9f4da49cc5bf6eae15df5c, State: Initialized, Role: FOLLOWER
I20260807 00:06:06.004171 31312 consensus_queue.cc:260] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.004935 31057 heartbeater.cc:503] Master 127.30.204.126:39819 was elected leader, sending a full tablet report...
I20260807 00:06:06.005774 31312 ts_tablet_manager.cc:1428] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Time spent starting tablet: real 0.007s user 0.006s sys 0.000s
W20260807 00:06:06.007695 30841 tablet.cc:2369] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260807 00:06:06.173883 31285 tablet.cc:2369] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:06:06.199119 31320 raft_consensus.cc:482] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260807 00:06:06.199389 31320 raft_consensus.cc:504] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.200750 31320 leader_election.cc:288] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 7c327a056abd434c8a5696dd5abd8e7b (127.30.204.67:38641), 5f6e5f1ae4334139a127b1816a2a4b46 (127.30.204.65:46389)
I20260807 00:06:06.204793 31302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:06.201189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:35181 (local address 127.30.204.67:38641)
0807 00:06:06.201324 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:06.201330 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:06.201590 (+ 260us) server_negotiation.cc:407] Connection header received
0807 00:06:06.201744 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:06.201746 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:06.201791 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:06.201854 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:06.202122 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.202623 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.203743 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.204178 (+ 435us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.204290 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:06.204341 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:06.204581 (+ 240us) server_negotiation.cc:299] Negotiation successful
0807 00:06:06.204654 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260807 00:06:06.204805 31304 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:06.201174 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.30.204.66:53745 (local address 127.30.204.65:46389)
0807 00:06:06.201315 (+ 141us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:06.201320 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:06.201432 (+ 112us) server_negotiation.cc:407] Connection header received
0807 00:06:06.201588 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:06.201591 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:06.201640 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:06.201715 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0807 00:06:06.202011 (+ 296us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.202531 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.203722 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.204137 (+ 415us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.204266 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:06.204328 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:06.204583 (+ 255us) server_negotiation.cc:299] Negotiation successful
0807 00:06:06.204660 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:06:06.204814 31059 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:06.201085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:46389 (local address 127.30.204.66:53745)
0807 00:06:06.201314 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:06.201332 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:06.201465 (+ 133us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:06.201739 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:06.201742 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:06.201760 (+ 18us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:06.201890 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:06.201898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.202660 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.202664 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:06.203579 (+ 915us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:06.203591 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.204267 (+ 676us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.204271 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:06.204441 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:06.204471 (+ 30us) client_negotiation.cc:769] Sending connection context
0807 00:06:06.204551 (+ 80us) client_negotiation.cc:240] Negotiation successful
0807 00:06:06.204644 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"mutex_wait_us":52}
I20260807 00:06:06.204819 31321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:06.201110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.67:38641 (local address 127.30.204.66:35181)
0807 00:06:06.201485 (+ 375us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:06.201497 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:06.201635 (+ 138us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:06.201873 (+ 238us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:06.201875 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:06.201891 (+ 16us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0807 00:06:06.202008 (+ 117us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:06.202014 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.202728 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.202732 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:06.203618 (+ 886us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:06.203629 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.204289 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.204293 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:06.204455 (+ 162us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:06.204482 (+ 27us) client_negotiation.cc:769] Sending connection context
0807 00:06:06.204549 (+ 67us) client_negotiation.cc:240] Negotiation successful
0807 00:06:06.204643 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"mutex_wait_us":42,"thread_start_us":96,"threads_started":1}
I20260807 00:06:06.205600 30761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fea0acd275413e9936b251874457a1" candidate_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" is_pre_election: true
I20260807 00:06:06.205711 31202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fea0acd275413e9936b251874457a1" candidate_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c327a056abd434c8a5696dd5abd8e7b" is_pre_election: true
I20260807 00:06:06.205951 30761 raft_consensus.cc:2454] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe39b4f9f9f4da49cc5bf6eae15df5c in term 0.
I20260807 00:06:06.205996 31202 raft_consensus.cc:2454] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate dfe39b4f9f9f4da49cc5bf6eae15df5c in term 0.
I20260807 00:06:06.206489 30867 leader_election.cc:302] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 7c327a056abd434c8a5696dd5abd8e7b, dfe39b4f9f9f4da49cc5bf6eae15df5c; no voters:
I20260807 00:06:06.206800 31320 raft_consensus.cc:2781] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260807 00:06:06.206902 31320 raft_consensus.cc:482] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260807 00:06:06.206959 31320 raft_consensus.cc:3037] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:06.208232 31320 raft_consensus.cc:504] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.208647 31320 leader_election.cc:288] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [CANDIDATE]: Term 1 election: Requested vote from peers 7c327a056abd434c8a5696dd5abd8e7b (127.30.204.67:38641), 5f6e5f1ae4334139a127b1816a2a4b46 (127.30.204.65:46389)
I20260807 00:06:06.209103 30761 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fea0acd275413e9936b251874457a1" candidate_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5f6e5f1ae4334139a127b1816a2a4b46"
I20260807 00:06:06.209112 31202 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "43fea0acd275413e9936b251874457a1" candidate_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "7c327a056abd434c8a5696dd5abd8e7b"
I20260807 00:06:06.209257 30761 raft_consensus.cc:3037] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:06.209257 31202 raft_consensus.cc:3037] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:06.210671 31202 raft_consensus.cc:2454] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe39b4f9f9f4da49cc5bf6eae15df5c in term 1.
I20260807 00:06:06.210671 30761 raft_consensus.cc:2454] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate dfe39b4f9f9f4da49cc5bf6eae15df5c in term 1.
I20260807 00:06:06.211069 30865 leader_election.cc:302] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5f6e5f1ae4334139a127b1816a2a4b46, dfe39b4f9f9f4da49cc5bf6eae15df5c; no voters:
I20260807 00:06:06.211278 31320 raft_consensus.cc:2781] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:06.211551 31320 raft_consensus.cc:686] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 1 LEADER]: Becoming Leader. State: Replica: dfe39b4f9f9f4da49cc5bf6eae15df5c, State: Running, Role: LEADER
W20260807 00:06:06.211719 31058 tablet.cc:2369] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260807 00:06:06.212178 31320 consensus_queue.cc:237] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } }
I20260807 00:06:06.215934 30527 catalog_manager.cc:5495] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c reported cstate change: term changed from 0 to 1, leader changed from <none> to dfe39b4f9f9f4da49cc5bf6eae15df5c (127.30.204.66). New cstate: current_term: 1 leader_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "dfe39b4f9f9f4da49cc5bf6eae15df5c" member_type: VOTER last_known_addr { host: "127.30.204.66" port: 40331 } health_report { overall_health: HEALTHY } } }
I20260807 00:06:06.279115 31307 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:06.275051 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54412 (local address 127.30.204.66:40331)
0807 00:06:06.275185 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:06.275189 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:06.275203 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:06:06.275333 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:06.275335 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:06.275397 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:06.275536 (+ 139us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:06.275773 (+ 237us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.276401 (+ 628us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.277395 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:06.277642 (+ 247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:06.277732 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:06.278104 (+ 372us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:06.278583 (+ 479us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:06.278687 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:06.278856 (+ 169us) server_negotiation.cc:299] Negotiation successful
0807 00:06:06.278921 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260807 00:06:06.285667 30761 raft_consensus.cc:1261] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 1 FOLLOWER]: Refusing update from remote peer dfe39b4f9f9f4da49cc5bf6eae15df5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:06.285667 31202 raft_consensus.cc:1261] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 1 FOLLOWER]: Refusing update from remote peer dfe39b4f9f9f4da49cc5bf6eae15df5c: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:06.286484 31322 consensus_queue.cc:1035] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "5f6e5f1ae4334139a127b1816a2a4b46" member_type: VOTER last_known_addr { host: "127.30.204.65" port: 46389 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:06.286682 31320 consensus_queue.cc:1035] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [LEADER]: Connected to new peer: Peer: permanent_uuid: "7c327a056abd434c8a5696dd5abd8e7b" member_type: VOTER last_known_addr { host: "127.30.204.67" port: 38641 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:06.304375 30527 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:47724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:06.304553 30527 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:47724:
table { table_name: "test-table" } modify_external_catalogs: true
I20260807 00:06:06.307129 30527 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 69f259c223c74dc8a50ad7deb08830d7: Sending DeleteTablet for 3 replicas of tablet 43fea0acd275413e9936b251874457a1
I20260807 00:06:06.308058 30944 tablet_service.cc:1514] Processing DeleteTablet for tablet 43fea0acd275413e9936b251874457a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:06 UTC) from {username='slave'} at 127.0.0.1:54396
I20260807 00:06:06.308056 31169 tablet_service.cc:1514] Processing DeleteTablet for tablet 43fea0acd275413e9936b251874457a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:06 UTC) from {username='slave'} at 127.0.0.1:43986
I20260807 00:06:06.308137 30728 tablet_service.cc:1514] Processing DeleteTablet for tablet 43fea0acd275413e9936b251874457a1 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-08-07 00:06:06 UTC) from {username='slave'} at 127.0.0.1:53966
I20260807 00:06:06.308692 31351 tablet_replica.cc:330] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46: stopping tablet replica
I20260807 00:06:06.309063 31351 raft_consensus.cc:2229] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:06:06.309484 31351 raft_consensus.cc:2258] T 43fea0acd275413e9936b251874457a1 P 5f6e5f1ae4334139a127b1816a2a4b46 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:06.309583 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 30630
I20260807 00:06:06.312530 31349 tablet_replica.cc:330] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: stopping tablet replica
I20260807 00:06:06.312973 31349 raft_consensus.cc:2229] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 1 LEADER]: Raft consensus shutting down.
I20260807 00:06:06.313452 31349 raft_consensus.cc:2258] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:06.314517 31347 tablet_replica.cc:330] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: stopping tablet replica
I20260807 00:06:06.314812 31347 raft_consensus.cc:2229] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260807 00:06:06.315090 31347 raft_consensus.cc:2258] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260807 00:06:06.317822 31349 ts_tablet_manager.cc:1905] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:06.317821 31347 ts_tablet_manager.cc:1905] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260807 00:06:06.320814 31349 ts_tablet_manager.cc:1918] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:06.320940 31349 log.cc:1196] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-1/wal/wals/43fea0acd275413e9936b251874457a1
I20260807 00:06:06.321058 31347 ts_tablet_manager.cc:1918] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260807 00:06:06.321167 31347 log.cc:1196] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Deleting WAL directory at /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1786060901439409-31537-0/minicluster-data/ts-2/wal/wals/43fea0acd275413e9936b251874457a1
I20260807 00:06:06.321255 31349 ts_tablet_manager.cc:1939] T 43fea0acd275413e9936b251874457a1 P dfe39b4f9f9f4da49cc5bf6eae15df5c: Deleting consensus metadata
I20260807 00:06:06.321477 31347 ts_tablet_manager.cc:1939] T 43fea0acd275413e9936b251874457a1 P 7c327a056abd434c8a5696dd5abd8e7b: Deleting consensus metadata
I20260807 00:06:06.322647 30501 catalog_manager.cc:4841] TS dfe39b4f9f9f4da49cc5bf6eae15df5c (127.30.204.66:40331): tablet 43fea0acd275413e9936b251874457a1 (table test-table [id=44dfd4e6f2154abe8376f11e4a27bf2a]) successfully deleted
I20260807 00:06:06.322736 30507 catalog_manager.cc:4841] TS 7c327a056abd434c8a5696dd5abd8e7b (127.30.204.67:38641): tablet 43fea0acd275413e9936b251874457a1 (table test-table [id=44dfd4e6f2154abe8376f11e4a27bf2a]) successfully deleted
W20260807 00:06:06.329093 30506 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.30.204.65:46389 (error 108)
I20260807 00:06:06.329329 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 30843
I20260807 00:06:06.332966 31306 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:06.332535 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.65:46389 (local address 127.0.0.1:53974)
0807 00:06:06.332754 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:06.332853 (+ 99us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.65:46389: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127}
W20260807 00:06:06.333396 30506 catalog_manager.cc:4584] TS 5f6e5f1ae4334139a127b1816a2a4b46 (127.30.204.65:46389): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 43fea0acd275413e9936b251874457a1: Network error: Client connection negotiation failed: client connection to 127.30.204.65:46389: connect: Connection refused (error 111)
I20260807 00:06:06.343628 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31061
I20260807 00:06:06.365157 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 30479
2026-08-07T00:06:06Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (1353 ms)
[----------] 6 tests from EncryptionPolicyTest (11881 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:06 dist-test-slave-pww0 krb5kdc[31356](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:06 dist-test-slave-pww0 krb5kdc[31356](info): set up 2 sockets
Aug 07 00:06:06 dist-test-slave-pww0 krb5kdc[31356](info): commencing operation
krb5kdc: starting...
W20260807 00:06:07.531605 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:07 dist-test-slave-pww0 krb5kdc[31356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:07.671270 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44419
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45283
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:44419
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:07.790584 31376 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:07.791036 31376 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:07.791118 31376 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:07.795806 31376 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:07.795885 31376 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:07.795905 31376 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:07.795921 31376 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:07.795935 31376 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:07.801692 31376 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45283
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:44419
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:44419
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31376
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:07.802832 31376 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:07.803937 31376 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:07.812837 31389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:07.812840 31383 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:07.812884 31376 server_base.cc:1034] running on GCE node
W20260807 00:06:07.812840 31386 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:07.813663 31376 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:07.814867 31376 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:07.816033 31376 hybrid_clock.cc:648] HybridClock initialized: now 1786061167816012 us; error 39 us; skew 500 ppm
Aug 07 00:06:07 dist-test-slave-pww0 krb5kdc[31356](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061167, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:07.823125 31376 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:07.824334 31376 webserver.cc:460] Webserver started at http://127.0.0.1:42767/ using document root <none> and password file <none>
I20260807 00:06:07.824959 31376 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:07.825039 31376 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:07.825246 31376 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:07.827620 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "599b135adef445b18279a2f686e07ee4"
format_stamp: "Formatted at 2026-08-07 00:06:07 on dist-test-slave-pww0"
server_key: "1115c3aec4621fc2b26079f88ba21a4c"
server_key_iv: "8e404bd6cc5fc65f8163d0630bea773b"
server_key_version: "encryptionkey@0"
I20260807 00:06:07.828243 31376 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "599b135adef445b18279a2f686e07ee4"
format_stamp: "Formatted at 2026-08-07 00:06:07 on dist-test-slave-pww0"
server_key: "1115c3aec4621fc2b26079f88ba21a4c"
server_key_iv: "8e404bd6cc5fc65f8163d0630bea773b"
server_key_version: "encryptionkey@0"
I20260807 00:06:07.832728 31376 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:07.835506 31402 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:07.836905 31376 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:07.837040 31376 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "599b135adef445b18279a2f686e07ee4"
format_stamp: "Formatted at 2026-08-07 00:06:07 on dist-test-slave-pww0"
server_key: "1115c3aec4621fc2b26079f88ba21a4c"
server_key_iv: "8e404bd6cc5fc65f8163d0630bea773b"
server_key_version: "encryptionkey@0"
I20260807 00:06:07.837170 31376 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:07.860852 31376 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:07.864426 31376 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:07.864637 31376 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:07.872417 31376 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:44419
I20260807 00:06:07.872447 31486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:44419 every 8 connection(s)
I20260807 00:06:07.873533 31376 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:07.876933 31487 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:07.879904 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31376
I20260807 00:06:07.880046 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:07.880352 31537 external_mini_cluster.cc:1428] Setting key 3b3fe984ee4835e8984a53d2a1883066
I20260807 00:06:07.884387 31487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4: Bootstrap starting.
I20260807 00:06:07.886835 31487 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:07.887702 31487 log.cc:824] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4: Log is configured to *not* fsync() on all Append() calls
Aug 07 00:06:07 dist-test-slave-pww0 krb5kdc[31356](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061167, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:06:07.889925 31487 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4: No bootstrap required, opened a new log
I20260807 00:06:07.893400 31487 raft_consensus.cc:348] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599b135adef445b18279a2f686e07ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44419 } }
I20260807 00:06:07.893682 31487 raft_consensus.cc:374] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:07.893755 31487 raft_consensus.cc:729] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 599b135adef445b18279a2f686e07ee4, State: Initialized, Role: FOLLOWER
I20260807 00:06:07.894246 31487 consensus_queue.cc:260] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599b135adef445b18279a2f686e07ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44419 } }
I20260807 00:06:07.894395 31487 raft_consensus.cc:388] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:07.894465 31487 raft_consensus.cc:482] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:07.894568 31487 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:07.895772 31487 raft_consensus.cc:504] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599b135adef445b18279a2f686e07ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44419 } }
I20260807 00:06:07.896157 31487 leader_election.cc:302] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 599b135adef445b18279a2f686e07ee4; no voters:
I20260807 00:06:07.896229 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:07.882093 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50371 (local address 127.0.0.1:44419)
0807 00:06:07.882622 (+ 529us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:07.882638 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:07.882680 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:06:07.883423 (+ 743us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:07.883445 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:07.883894 (+ 449us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:07.884237 (+ 343us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:07.885135 (+ 898us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:07.886195 (+ 1060us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:07.887165 (+ 970us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:07.887740 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:07.887902 (+ 162us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:07.890591 (+ 2689us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:07.890597 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:07.890613 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:07.890646 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:07.892861 (+ 2215us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:07.893464 (+ 603us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:07.893470 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:07.893475 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:07.893608 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:07.894000 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:07.894003 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:07.894004 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:07.894464 (+ 460us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:07.894686 (+ 222us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:07.895034 (+ 348us) server_negotiation.cc:299] Negotiation successful
0807 00:06:07.895277 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":362,"thread_start_us":159,"threads_started":1}
I20260807 00:06:07.896502 31487 leader_election.cc:288] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:07.896617 31496 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:07.896831 31496 raft_consensus.cc:686] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [term 1 LEADER]: Becoming Leader. State: Replica: 599b135adef445b18279a2f686e07ee4, State: Running, Role: LEADER
I20260807 00:06:07.897157 31496 consensus_queue.cc:237] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599b135adef445b18279a2f686e07ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44419 } }
I20260807 00:06:07.897958 31487 sys_catalog.cc:564] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:07.899089 31501 sys_catalog.cc:455] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 599b135adef445b18279a2f686e07ee4. Latest consensus state: current_term: 1 leader_uuid: "599b135adef445b18279a2f686e07ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599b135adef445b18279a2f686e07ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44419 } } }
I20260807 00:06:07.899098 31500 sys_catalog.cc:455] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "599b135adef445b18279a2f686e07ee4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "599b135adef445b18279a2f686e07ee4" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 44419 } } }
I20260807 00:06:07.899246 31501 sys_catalog.cc:458] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:07.899255 31500 sys_catalog.cc:458] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:07.899955 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:07.899972 31511 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:07.903378 31511 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:07.911428 31493 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:07.902237 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59601 (local address 127.0.0.1:44419)
0807 00:06:07.902350 (+ 113us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:07.902354 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:07.902646 (+ 292us) server_negotiation.cc:407] Connection header received
0807 00:06:07.902773 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:07.902776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:07.902821 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:07.902882 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:07.903670 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:07.904198 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:07.905013 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:07.905279 (+ 266us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:07.905370 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:07.907202 (+ 1832us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:07.907207 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:07.907210 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:07.907250 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:07.909008 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:07.909700 (+ 692us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:07.909704 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:07.909705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:07.909806 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:07.910275 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:07.910279 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:07.910281 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:07.910540 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:07.910682 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:07.910991 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 00:06:07.911093 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:06:07.912410 31511 catalog_manager.cc:1347] Generated new cluster ID: c608ba8157d5429384cc8b1f6bd2570f
I20260807 00:06:07.912478 31511 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:07.922315 31511 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:07.923535 31511 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:07.933785 31511 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 599b135adef445b18279a2f686e07ee4: Generated new TSK 0
I20260807 00:06:07.934597 31511 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:07.944101 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31376
W20260807 00:06:07.962675 31354 connection.cc:535] client connection to 127.0.0.1:44419 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:44419): Connection reset by peer (error 104)
2026-08-07T00:06:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1587 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:08 dist-test-slave-pww0 krb5kdc[31552](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:08 dist-test-slave-pww0 krb5kdc[31552](info): set up 2 sockets
Aug 07 00:06:08 dist-test-slave-pww0 krb5kdc[31552](info): commencing operation
krb5kdc: starting...
W20260807 00:06:09.109612 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.001s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:09 dist-test-slave-pww0 krb5kdc[31552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061169, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:09.244477 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42303
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:42129
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42303
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:09.364933 31568 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:09.365254 31568 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:09.365319 31568 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:09.369484 31568 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:09.369565 31568 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:09.369601 31568 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:09.369624 31568 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:09.369652 31568 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:09.375278 31568 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:42129
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42303
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42303
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31568
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:09.376545 31568 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:09.377676 31568 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:09.386796 31579 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:09.386868 31568 server_base.cc:1034] running on GCE node
W20260807 00:06:09.386785 31577 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:09.386781 31575 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:09.387728 31568 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:09.388978 31568 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:09.390141 31568 hybrid_clock.cc:648] HybridClock initialized: now 1786061169390125 us; error 35 us; skew 500 ppm
Aug 07 00:06:09 dist-test-slave-pww0 krb5kdc[31552](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061169, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:09.393191 31568 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:09.394187 31568 webserver.cc:460] Webserver started at http://127.0.0.1:46109/ using document root <none> and password file <none>
I20260807 00:06:09.394781 31568 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:09.394848 31568 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:09.395100 31568 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:09.397433 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "e73624f7663a43998d3f42ffd4781cab"
format_stamp: "Formatted at 2026-08-07 00:06:09 on dist-test-slave-pww0"
server_key: "63b4f86b2628fb5744ab4fb728ff04e3"
server_key_iv: "e2629eb054a1ea8990fb9b3ccf5d4b45"
server_key_version: "encryptionkey@0"
I20260807 00:06:09.398029 31568 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "e73624f7663a43998d3f42ffd4781cab"
format_stamp: "Formatted at 2026-08-07 00:06:09 on dist-test-slave-pww0"
server_key: "63b4f86b2628fb5744ab4fb728ff04e3"
server_key_iv: "e2629eb054a1ea8990fb9b3ccf5d4b45"
server_key_version: "encryptionkey@0"
I20260807 00:06:09.402190 31568 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:06:09.404906 31588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:09.406271 31568 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260807 00:06:09.406407 31568 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "e73624f7663a43998d3f42ffd4781cab"
format_stamp: "Formatted at 2026-08-07 00:06:09 on dist-test-slave-pww0"
server_key: "63b4f86b2628fb5744ab4fb728ff04e3"
server_key_iv: "e2629eb054a1ea8990fb9b3ccf5d4b45"
server_key_version: "encryptionkey@0"
I20260807 00:06:09.406525 31568 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:09.430284 31568 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:09.433620 31568 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:09.433840 31568 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:09.441756 31568 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42303
I20260807 00:06:09.441779 31677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42303 every 8 connection(s)
I20260807 00:06:09.442869 31568 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:09.446102 31678 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:09.452296 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab: Bootstrap starting.
I20260807 00:06:09.452690 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31568
I20260807 00:06:09.452813 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:09.453089 31537 external_mini_cluster.cc:1428] Setting key 499ed2410c02d17d6e81659d02d52ec9
I20260807 00:06:09.455145 31678 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:09.455968 31678 log.cc:824] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:09.458019 31678 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab: No bootstrap required, opened a new log
I20260807 00:06:09.461920 31678 raft_consensus.cc:348] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73624f7663a43998d3f42ffd4781cab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42303 } }
I20260807 00:06:09.462180 31678 raft_consensus.cc:374] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:09.462240 31678 raft_consensus.cc:729] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e73624f7663a43998d3f42ffd4781cab, State: Initialized, Role: FOLLOWER
I20260807 00:06:09.462262 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:09.454449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37901 (local address 127.0.0.1:42303)
0807 00:06:09.454938 (+ 489us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:09.454954 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:09.454993 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:06:09.455766 (+ 773us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:09.455788 (+ 22us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:09.456190 (+ 402us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:09.456522 (+ 332us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:09.456958 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:09.457935 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:09.458868 (+ 933us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:09.459353 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:09.459538 (+ 185us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:09.460468 (+ 930us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:09.460473 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:09.460487 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:09.460903 (+ 416us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:09.460962 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:09.460986 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:09.461105 (+ 119us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:09.461305 (+ 200us) server_negotiation.cc:299] Negotiation successful
0807 00:06:09.461455 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":324,"thread_start_us":144,"threads_started":1}
I20260807 00:06:09.462776 31678 consensus_queue.cc:260] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73624f7663a43998d3f42ffd4781cab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42303 } }
I20260807 00:06:09.462952 31678 raft_consensus.cc:388] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:09.463016 31678 raft_consensus.cc:482] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:09.463112 31678 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:09.464185 31678 raft_consensus.cc:504] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73624f7663a43998d3f42ffd4781cab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42303 } }
I20260807 00:06:09.464572 31678 leader_election.cc:302] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e73624f7663a43998d3f42ffd4781cab; no voters:
I20260807 00:06:09.464927 31678 leader_election.cc:288] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:09.465018 31685 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:09.465256 31685 raft_consensus.cc:686] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [term 1 LEADER]: Becoming Leader. State: Replica: e73624f7663a43998d3f42ffd4781cab, State: Running, Role: LEADER
I20260807 00:06:09.465591 31685 consensus_queue.cc:237] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73624f7663a43998d3f42ffd4781cab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42303 } }
I20260807 00:06:09.465632 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:09.466372 31678 sys_catalog.cc:564] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:09.467310 31687 sys_catalog.cc:455] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e73624f7663a43998d3f42ffd4781cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73624f7663a43998d3f42ffd4781cab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42303 } } }
I20260807 00:06:09.467310 31690 sys_catalog.cc:455] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [sys.catalog]: SysCatalogTable state changed. Reason: New leader e73624f7663a43998d3f42ffd4781cab. Latest consensus state: current_term: 1 leader_uuid: "e73624f7663a43998d3f42ffd4781cab" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e73624f7663a43998d3f42ffd4781cab" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42303 } } }
I20260807 00:06:09.467492 31687 sys_catalog.cc:458] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:09.467542 31690 sys_catalog.cc:458] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab [sys.catalog]: This master's current role is: LEADER
W20260807 00:06:09.471659 31723 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:09.471753 31723 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:09.471840 31715 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:09.472620 31715 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:09.474738 31683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:09.468077 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51883 (local address 127.0.0.1:42303)
0807 00:06:09.468204 (+ 127us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:09.468209 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:09.468633 (+ 424us) server_negotiation.cc:407] Connection header received
0807 00:06:09.468843 (+ 210us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:09.468847 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:09.468898 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:09.468951 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:09.469455 (+ 504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:09.470122 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:09.472517 (+ 2395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:09.472869 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:09.472972 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:09.474154 (+ 1182us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:09.474158 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:09.474161 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:09.474178 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:09.474210 (+ 32us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:09.474234 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:09.474317 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:09.474552 (+ 235us) server_negotiation.cc:299] Negotiation successful
0807 00:06:09.474598 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260807 00:06:09.478549 31715 catalog_manager.cc:1347] Generated new cluster ID: cdd1f9a3fdc54312b2b41575f40da41c
I20260807 00:06:09.478631 31715 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:09.485105 31715 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:09.486420 31715 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:09.496997 31715 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e73624f7663a43998d3f42ffd4781cab: Generated new TSK 0
I20260807 00:06:09.497758 31715 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:09.503367 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31568
W20260807 00:06:09.524720 31550 connection.cc:535] client connection to 127.0.0.1:42303 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42303): Connection reset by peer (error 104)
2026-08-07T00:06:09Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1561 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:09 dist-test-slave-pww0 krb5kdc[31733](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:09 dist-test-slave-pww0 krb5kdc[31733](info): set up 2 sockets
Aug 07 00:06:09 dist-test-slave-pww0 krb5kdc[31733](info): commencing operation
krb5kdc: starting...
W20260807 00:06:10.672127 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.133s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:10 dist-test-slave-pww0 krb5kdc[31733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061170, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:10.808749 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37033
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37663
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37033
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:10.925401 31756 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:10.925719 31756 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:10.925781 31756 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:10.929919 31756 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:10.930007 31756 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:10.930035 31756 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:10.930056 31756 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:10.930076 31756 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:10.935792 31756 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37663
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37033
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37033
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31756
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:10.936951 31756 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:10.938045 31756 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:10.947371 31772 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:10.947364 31766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:10.947364 31765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:10.947440 31756 server_base.cc:1034] running on GCE node
I20260807 00:06:10.948253 31756 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:10.949461 31756 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:10.950645 31756 hybrid_clock.cc:648] HybridClock initialized: now 1786061170950632 us; error 33 us; skew 500 ppm
Aug 07 00:06:10 dist-test-slave-pww0 krb5kdc[31733](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061170, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:10.953840 31756 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:10.954897 31756 webserver.cc:460] Webserver started at http://127.0.0.1:40629/ using document root <none> and password file <none>
I20260807 00:06:10.955533 31756 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:10.955613 31756 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:10.955860 31756 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:10.958038 31756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "729c9f26a47d4ea08d2ebcf037c7cd78"
format_stamp: "Formatted at 2026-08-07 00:06:10 on dist-test-slave-pww0"
server_key: "2a366344984bf70f40c07279fc921794"
server_key_iv: "ef9bddbefaf390ebc2b1831d461ae321"
server_key_version: "encryptionkey@0"
I20260807 00:06:10.958612 31756 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "729c9f26a47d4ea08d2ebcf037c7cd78"
format_stamp: "Formatted at 2026-08-07 00:06:10 on dist-test-slave-pww0"
server_key: "2a366344984bf70f40c07279fc921794"
server_key_iv: "ef9bddbefaf390ebc2b1831d461ae321"
server_key_version: "encryptionkey@0"
I20260807 00:06:10.962590 31756 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:06:10.965274 31781 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:10.966683 31756 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:10.966873 31756 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "729c9f26a47d4ea08d2ebcf037c7cd78"
format_stamp: "Formatted at 2026-08-07 00:06:10 on dist-test-slave-pww0"
server_key: "2a366344984bf70f40c07279fc921794"
server_key_iv: "ef9bddbefaf390ebc2b1831d461ae321"
server_key_version: "encryptionkey@0"
I20260807 00:06:10.966986 31756 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:10.971899 31756 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:10.975207 31756 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:10.975404 31756 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:10.982863 31756 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37033
I20260807 00:06:10.982892 31861 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37033 every 8 connection(s)
I20260807 00:06:10.983999 31756 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:10.986634 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31756
I20260807 00:06:10.986791 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:10.987135 31537 external_mini_cluster.cc:1428] Setting key 001c496eb261dd256aea5853d6b83dbe
I20260807 00:06:10.987403 31862 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:10.992939 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:10.988722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39067 (local address 127.0.0.1:37033)
0807 00:06:10.989235 (+ 513us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:10.989260 (+ 25us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:10.989299 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:06:10.990208 (+ 909us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:10.990239 (+ 31us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:10.990559 (+ 320us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:10.990892 (+ 333us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:10.991723 (+ 831us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:10.991729 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:10.991745 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:10.991756 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:10.991815 (+ 59us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:10.991835 (+ 20us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:10.991967 (+ 132us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:10.992047 (+ 80us) server_negotiation.cc:299] Negotiation successful
0807 00:06:10.992212 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":153,"threads_started":1}
I20260807 00:06:10.994585 31862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78: Bootstrap starting.
I20260807 00:06:10.996026 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:10.997385 31862 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:10.998246 31862 log.cc:824] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:11.000051 31867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:10.998345 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40285 (local address 127.0.0.1:37033)
0807 00:06:10.998468 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:10.998472 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:10.998729 (+ 257us) server_negotiation.cc:407] Connection header received
0807 00:06:10.998876 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:10.998879 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:10.998916 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:10.998980 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:10.999677 (+ 697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:10.999681 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:10.999684 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:10.999695 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:10.999723 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:10.999737 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:10.999811 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:10.999893 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 00:06:10.999930 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:06:11.000548 31862 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78: No bootstrap required, opened a new log
I20260807 00:06:11.004531 31862 raft_consensus.cc:348] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260807 00:06:11.004808 31862 raft_consensus.cc:374] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:11.004881 31862 raft_consensus.cc:729] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 729c9f26a47d4ea08d2ebcf037c7cd78, State: Initialized, Role: FOLLOWER
I20260807 00:06:11.005316 31862 consensus_queue.cc:260] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260807 00:06:11.005461 31862 raft_consensus.cc:388] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:11.005528 31862 raft_consensus.cc:482] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:11.005640 31862 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:11.006680 31862 raft_consensus.cc:504] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260807 00:06:11.007028 31862 leader_election.cc:302] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 729c9f26a47d4ea08d2ebcf037c7cd78; no voters:
I20260807 00:06:11.007364 31862 leader_election.cc:288] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:11.007625 31877 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:11.008167 31877 raft_consensus.cc:686] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [term 1 LEADER]: Becoming Leader. State: Replica: 729c9f26a47d4ea08d2ebcf037c7cd78, State: Running, Role: LEADER
I20260807 00:06:11.008651 31877 consensus_queue.cc:237] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } }
I20260807 00:06:11.008728 31862 sys_catalog.cc:564] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:11.010427 31879 sys_catalog.cc:455] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 729c9f26a47d4ea08d2ebcf037c7cd78. Latest consensus state: current_term: 1 leader_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } } }
I20260807 00:06:11.010597 31879 sys_catalog.cc:458] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:11.010911 31878 sys_catalog.cc:455] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "729c9f26a47d4ea08d2ebcf037c7cd78" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37033 } } }
I20260807 00:06:11.011196 31878 sys_catalog.cc:458] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78 [sys.catalog]: This master's current role is: LEADER
W20260807 00:06:11.013702 31901 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:11.013777 31901 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:11.013852 31899 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:11.014577 31899 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:11.020815 31899 catalog_manager.cc:1347] Generated new cluster ID: 1ac8f710a812462799fb1d80d7c8d7b5
I20260807 00:06:11.020906 31899 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:11.030766 31899 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:11.031144 31899 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:11.037843 31899 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 729c9f26a47d4ea08d2ebcf037c7cd78: Generated new TSK 0
I20260807 00:06:11.038792 31899 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:11.052323 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31756
2026-08-07T00:06:11Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1541 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:11 dist-test-slave-pww0 krb5kdc[31908](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:11 dist-test-slave-pww0 krb5kdc[31908](info): set up 2 sockets
Aug 07 00:06:11 dist-test-slave-pww0 krb5kdc[31908](info): commencing operation
krb5kdc: starting...
W20260807 00:06:12.207823 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.127s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:12 dist-test-slave-pww0 krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:12.342959 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35565
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38757
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:35565
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:12.462133 31930 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:12.462455 31930 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:12.462499 31930 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:12.466720 31930 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:12.466812 31930 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:12.466830 31930 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:12.466846 31930 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:12.466861 31930 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:12.472857 31930 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38757
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:35565
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:35565
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31930
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:12.474062 31930 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:12.475178 31930 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:12.484203 31936 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:12.484201 31942 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:12.484313 31930 server_base.cc:1034] running on GCE node
W20260807 00:06:12.484201 31935 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:12.485138 31930 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:12.486368 31930 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:12.487567 31930 hybrid_clock.cc:648] HybridClock initialized: now 1786061172487551 us; error 29 us; skew 500 ppm
Aug 07 00:06:12 dist-test-slave-pww0 krb5kdc[31908](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061172, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:12.490587 31930 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:12.491782 31930 webserver.cc:460] Webserver started at http://127.0.0.1:41141/ using document root <none> and password file <none>
I20260807 00:06:12.492365 31930 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:12.492441 31930 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:12.492663 31930 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:12.494887 31930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "2696840317fc49bc9fc29d7eb5a0c67f"
format_stamp: "Formatted at 2026-08-07 00:06:12 on dist-test-slave-pww0"
server_key: "d09419966e3b200b1de294d8597c6920"
server_key_iv: "8b572ae9b9d2239ba33171adcbb4b85d"
server_key_version: "encryptionkey@0"
I20260807 00:06:12.495522 31930 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "2696840317fc49bc9fc29d7eb5a0c67f"
format_stamp: "Formatted at 2026-08-07 00:06:12 on dist-test-slave-pww0"
server_key: "d09419966e3b200b1de294d8597c6920"
server_key_iv: "8b572ae9b9d2239ba33171adcbb4b85d"
server_key_version: "encryptionkey@0"
I20260807 00:06:12.499634 31930 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:12.502398 31951 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:12.503804 31930 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:12.503937 31930 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "2696840317fc49bc9fc29d7eb5a0c67f"
format_stamp: "Formatted at 2026-08-07 00:06:12 on dist-test-slave-pww0"
server_key: "d09419966e3b200b1de294d8597c6920"
server_key_iv: "8b572ae9b9d2239ba33171adcbb4b85d"
server_key_version: "encryptionkey@0"
I20260807 00:06:12.504050 31930 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:12.518268 31930 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:12.521705 31930 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:12.521916 31930 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:12.529445 31930 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:35565
I20260807 00:06:12.529474 32027 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:35565 every 8 connection(s)
I20260807 00:06:12.530570 31930 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:12.531116 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 31930
I20260807 00:06:12.531277 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:12.531651 31537 external_mini_cluster.cc:1428] Setting key fabe33bc44110a2137c8bef27356430a
I20260807 00:06:12.533949 32028 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:12.541169 32028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f: Bootstrap starting.
Aug 07 00:06:12 dist-test-slave-pww0 krb5kdc[31908](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061172, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:06:12.544037 32028 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:12.545027 32028 log.cc:824] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:12.547160 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:12.533210 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60531 (local address 127.0.0.1:35565)
0807 00:06:12.533658 (+ 448us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:12.533676 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:12.533715 (+ 39us) server_negotiation.cc:407] Connection header received
0807 00:06:12.534409 (+ 694us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:12.534429 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:12.534805 (+ 376us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:12.535121 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:12.536126 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:12.537191 (+ 1065us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:12.538420 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:12.538921 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:12.539042 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:12.542082 (+ 3040us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:12.542088 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:12.542102 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:12.542136 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:12.544020 (+ 1884us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:12.544615 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:12.544621 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:12.544626 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:12.544744 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:12.545035 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:12.545038 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:12.545039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:12.545532 (+ 493us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:12.545734 (+ 202us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:12.546018 (+ 284us) server_negotiation.cc:299] Negotiation successful
0807 00:06:12.546336 (+ 318us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":298,"thread_start_us":134,"threads_started":1}
I20260807 00:06:12.547410 32028 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f: No bootstrap required, opened a new log
I20260807 00:06:12.550323 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:12.551566 32028 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35565 } }
I20260807 00:06:12.551937 32028 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:12.552058 32028 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2696840317fc49bc9fc29d7eb5a0c67f, State: Initialized, Role: FOLLOWER
I20260807 00:06:12.552608 32028 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35565 } }
I20260807 00:06:12.552775 32028 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:12.552881 32028 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:12.552994 32028 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:12.554406 32028 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35565 } }
I20260807 00:06:12.554828 32028 leader_election.cc:302] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2696840317fc49bc9fc29d7eb5a0c67f; no voters:
I20260807 00:06:12.555176 32028 leader_election.cc:288] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:12.555274 32041 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:12.555532 32041 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [term 1 LEADER]: Becoming Leader. State: Replica: 2696840317fc49bc9fc29d7eb5a0c67f, State: Running, Role: LEADER
I20260807 00:06:12.555943 32041 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35565 } }
I20260807 00:06:12.556777 32028 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:12.557660 32041 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2696840317fc49bc9fc29d7eb5a0c67f. Latest consensus state: current_term: 1 leader_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35565 } } }
I20260807 00:06:12.558487 32041 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:12.558354 32043 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2696840317fc49bc9fc29d7eb5a0c67f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 35565 } } }
I20260807 00:06:12.561005 32043 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:12.562282 32033 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:12.552907 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44973 (local address 127.0.0.1:35565)
0807 00:06:12.553037 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:12.553041 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:12.553268 (+ 227us) server_negotiation.cc:407] Connection header received
0807 00:06:12.553409 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:12.553411 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:12.553455 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:12.553520 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:12.554397 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:12.555070 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:12.556192 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:12.556445 (+ 253us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:12.556577 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:12.558233 (+ 1656us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:12.558237 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:12.558240 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:12.558270 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:12.559996 (+ 1726us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:12.560724 (+ 728us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:12.560728 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:12.560729 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:12.560825 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:12.561176 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:12.561179 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:12.561181 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:12.561424 (+ 243us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:12.561533 (+ 109us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:12.561944 (+ 411us) server_negotiation.cc:299] Negotiation successful
0807 00:06:12.562103 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 00:06:12.564862 32072 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:12.564960 32072 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:12.565024 32071 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:12.566005 32071 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:12.572708 32071 catalog_manager.cc:1347] Generated new cluster ID: d2014b92a8bf4babab51fa22b116745d
I20260807 00:06:12.572880 32071 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:12.604404 32071 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:12.605407 32071 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:12.614543 32071 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2696840317fc49bc9fc29d7eb5a0c67f: Generated new TSK 0
I20260807 00:06:12.615309 32071 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:12.627496 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 31930
2026-08-07T00:06:12Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1574 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:12 dist-test-slave-pww0 krb5kdc[32081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:12 dist-test-slave-pww0 krb5kdc[32081](info): set up 2 sockets
Aug 07 00:06:12 dist-test-slave-pww0 krb5kdc[32081](info): commencing operation
krb5kdc: starting...
W20260807 00:06:13.789347 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:13 dist-test-slave-pww0 krb5kdc[32081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061173, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:13Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:13Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:13.922816 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42921
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:34973
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42921
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:14.043098 32096 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:14.043422 32096 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:14.043495 32096 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:14.047821 32096 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:14.047909 32096 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:14.047927 32096 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:14.047942 32096 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:14.047958 32096 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:14.053857 32096 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:34973
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42921
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42921
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32096
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:14.055094 32096 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:14.056267 32096 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:14.065963 32103 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:14.066332 32096 server_base.cc:1034] running on GCE node
W20260807 00:06:14.066469 32104 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:14.065974 32108 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:14.067174 32096 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:14.068547 32096 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:14.069778 32096 hybrid_clock.cc:648] HybridClock initialized: now 1786061174069752 us; error 41 us; skew 500 ppm
Aug 07 00:06:14 dist-test-slave-pww0 krb5kdc[32081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061174, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:14.077621 32096 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:14.078835 32096 webserver.cc:460] Webserver started at http://127.0.0.1:35793/ using document root <none> and password file <none>
I20260807 00:06:14.079527 32096 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:14.079617 32096 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:14.079855 32096 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:14.082134 32096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "464976faf9d04970892b7b9cf69bd17a"
format_stamp: "Formatted at 2026-08-07 00:06:14 on dist-test-slave-pww0"
server_key: "eb4d87d439f4ca6178b2a64bd1a1fdf6"
server_key_iv: "9ba53dd45e27e69eb67346cf4d4ee39a"
server_key_version: "encryptionkey@0"
I20260807 00:06:14.082775 32096 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "464976faf9d04970892b7b9cf69bd17a"
format_stamp: "Formatted at 2026-08-07 00:06:14 on dist-test-slave-pww0"
server_key: "eb4d87d439f4ca6178b2a64bd1a1fdf6"
server_key_iv: "9ba53dd45e27e69eb67346cf4d4ee39a"
server_key_version: "encryptionkey@0"
I20260807 00:06:14.086956 32096 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:06:14.089874 32121 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:14.091254 32096 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:14.091393 32096 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "464976faf9d04970892b7b9cf69bd17a"
format_stamp: "Formatted at 2026-08-07 00:06:14 on dist-test-slave-pww0"
server_key: "eb4d87d439f4ca6178b2a64bd1a1fdf6"
server_key_iv: "9ba53dd45e27e69eb67346cf4d4ee39a"
server_key_version: "encryptionkey@0"
I20260807 00:06:14.091543 32096 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:14.107952 32096 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:14.111481 32096 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:14.111716 32096 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:14.119277 32096 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42921
I20260807 00:06:14.119300 32198 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42921 every 8 connection(s)
I20260807 00:06:14.120353 32096 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:14.121012 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 32096
I20260807 00:06:14.121140 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:14.121407 31537 external_mini_cluster.cc:1428] Setting key c167adfe13dee04b52988c61fb8bd7dc
I20260807 00:06:14.123732 32201 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:14.131209 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a: Bootstrap starting.
I20260807 00:06:14.132269 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:14.122872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59195 (local address 127.0.0.1:42921)
0807 00:06:14.123327 (+ 455us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:14.123348 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:14.123394 (+ 46us) server_negotiation.cc:407] Connection header received
0807 00:06:14.124288 (+ 894us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:14.124313 (+ 25us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:14.124770 (+ 457us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:14.125173 (+ 403us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:14.125614 (+ 441us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:14.127177 (+ 1563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:14.128268 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:14.128977 (+ 709us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:14.129171 (+ 194us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:14.130053 (+ 882us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:14.130059 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:14.130079 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:14.130633 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:14.130713 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:14.130746 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:14.130846 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:14.131007 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:06:14.131247 (+ 240us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":299,"thread_start_us":113,"threads_started":1}
I20260807 00:06:14.134088 32201 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:14.135006 32201 log.cc:824] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:14.135830 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:14.137486 32201 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a: No bootstrap required, opened a new log
I20260807 00:06:14.141520 32201 raft_consensus.cc:348] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464976faf9d04970892b7b9cf69bd17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260807 00:06:14.141804 32201 raft_consensus.cc:374] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:14.141872 32201 raft_consensus.cc:729] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 464976faf9d04970892b7b9cf69bd17a, State: Initialized, Role: FOLLOWER
I20260807 00:06:14.142172 32204 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:14.138240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53095 (local address 127.0.0.1:42921)
0807 00:06:14.138362 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:14.138367 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:14.138588 (+ 221us) server_negotiation.cc:407] Connection header received
0807 00:06:14.138711 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:14.138713 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:14.138755 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:14.138816 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:14.139099 (+ 283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:14.139683 (+ 584us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:14.140516 (+ 833us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:14.140748 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:14.140837 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:14.141635 (+ 798us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:14.141638 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:14.141640 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:14.141653 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:14.141680 (+ 27us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:14.141691 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:14.141749 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:14.141956 (+ 207us) server_negotiation.cc:299] Negotiation successful
0807 00:06:14.142028 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260807 00:06:14.142409 32201 consensus_queue.cc:260] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464976faf9d04970892b7b9cf69bd17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260807 00:06:14.142571 32201 raft_consensus.cc:388] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:14.142649 32201 raft_consensus.cc:482] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:14.142756 32201 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:14.143980 32201 raft_consensus.cc:504] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464976faf9d04970892b7b9cf69bd17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260807 00:06:14.144325 32201 leader_election.cc:302] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 464976faf9d04970892b7b9cf69bd17a; no voters:
I20260807 00:06:14.144626 32201 leader_election.cc:288] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:14.144750 32215 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:14.145013 32215 raft_consensus.cc:686] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [term 1 LEADER]: Becoming Leader. State: Replica: 464976faf9d04970892b7b9cf69bd17a, State: Running, Role: LEADER
I20260807 00:06:14.145351 32215 consensus_queue.cc:237] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464976faf9d04970892b7b9cf69bd17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } }
I20260807 00:06:14.145965 32201 sys_catalog.cc:564] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:14.147341 32218 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 464976faf9d04970892b7b9cf69bd17a. Latest consensus state: current_term: 1 leader_uuid: "464976faf9d04970892b7b9cf69bd17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464976faf9d04970892b7b9cf69bd17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } } }
I20260807 00:06:14.147358 32217 sys_catalog.cc:455] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "464976faf9d04970892b7b9cf69bd17a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "464976faf9d04970892b7b9cf69bd17a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42921 } } }
I20260807 00:06:14.147537 32217 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:14.148007 32227 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:14.147537 32218 sys_catalog.cc:458] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:14.151026 32227 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:14.157208 32227 catalog_manager.cc:1347] Generated new cluster ID: 6f59c9063a2045acbc822883cace18d1
I20260807 00:06:14.157310 32227 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:14.178596 32227 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:14.180092 32227 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:14.189716 32227 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 464976faf9d04970892b7b9cf69bd17a: Generated new TSK 0
I20260807 00:06:14.190711 32227 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:14.199417 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 32096
2026-08-07T00:06:14Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:14 dist-test-slave-pww0 krb5kdc[32246](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:14 dist-test-slave-pww0 krb5kdc[32246](info): set up 2 sockets
Aug 07 00:06:14 dist-test-slave-pww0 krb5kdc[32246](info): commencing operation
krb5kdc: starting...
W20260807 00:06:15.371398 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:15 dist-test-slave-pww0 krb5kdc[32246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061175, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:15.511008 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34971
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34971
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:15.633654 32267 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:15.633977 32267 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:15.634042 32267 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:15.638375 32267 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:15.638464 32267 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:15.638492 32267 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:15.638513 32267 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:15.638535 32267 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:15.644368 32267 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34971
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34971
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32267
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:15.645591 32267 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:15.646724 32267 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:15.656054 32278 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:15.656047 32276 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:15.656116 32267 server_base.cc:1034] running on GCE node
W20260807 00:06:15.656049 32274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:15.656953 32267 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:15.658196 32267 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:15.659412 32267 hybrid_clock.cc:648] HybridClock initialized: now 1786061175659387 us; error 45 us; skew 500 ppm
Aug 07 00:06:15 dist-test-slave-pww0 krb5kdc[32246](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061175, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:15.662523 32267 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:15.663600 32267 webserver.cc:460] Webserver started at http://127.0.0.1:34375/ using document root <none> and password file <none>
I20260807 00:06:15.664167 32267 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:15.664237 32267 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:15.664418 32267 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:15.666613 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "261e70f9ff234bf5a0cfe7d3623ec132"
format_stamp: "Formatted at 2026-08-07 00:06:15 on dist-test-slave-pww0"
server_key: "d35d9d330f41e1d46baff16b94398142"
server_key_iv: "a254ccfb4fdeaf4f971f64bb957fb610"
server_key_version: "encryptionkey@0"
I20260807 00:06:15.667169 32267 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "261e70f9ff234bf5a0cfe7d3623ec132"
format_stamp: "Formatted at 2026-08-07 00:06:15 on dist-test-slave-pww0"
server_key: "d35d9d330f41e1d46baff16b94398142"
server_key_iv: "a254ccfb4fdeaf4f971f64bb957fb610"
server_key_version: "encryptionkey@0"
I20260807 00:06:15.671128 32267 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.003s
I20260807 00:06:15.673786 32293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:15.675203 32267 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:15.675348 32267 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "261e70f9ff234bf5a0cfe7d3623ec132"
format_stamp: "Formatted at 2026-08-07 00:06:15 on dist-test-slave-pww0"
server_key: "d35d9d330f41e1d46baff16b94398142"
server_key_iv: "a254ccfb4fdeaf4f971f64bb957fb610"
server_key_version: "encryptionkey@0"
I20260807 00:06:15.675498 32267 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:15.680275 32267 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:15.683380 32267 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:15.683591 32267 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:15.690799 32267 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34971
I20260807 00:06:15.690811 32372 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34971 every 8 connection(s)
I20260807 00:06:15.691814 32267 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:15.694679 32373 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:15.699333 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 32267
I20260807 00:06:15.699508 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:15.699855 31537 external_mini_cluster.cc:1428] Setting key f977b719256bcbfe4185db41be13ab68
I20260807 00:06:15.701113 32373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132: Bootstrap starting.
I20260807 00:06:15.703611 32373 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:15.704432 32373 log.cc:824] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:15.705343 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:15.701411 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46593 (local address 127.0.0.1:34971)
0807 00:06:15.701923 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:15.701938 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:15.701974 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:06:15.702641 (+ 667us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:15.702659 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:15.702966 (+ 307us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:15.703289 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:15.704108 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:15.704113 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:15.704125 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:15.704135 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:15.704191 (+ 56us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:15.704212 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:15.704309 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:15.704434 (+ 125us) server_negotiation.cc:299] Negotiation successful
0807 00:06:15.704603 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":347,"thread_start_us":159,"threads_started":1}
I20260807 00:06:15.706560 32373 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132: No bootstrap required, opened a new log
I20260807 00:06:15.708401 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:15.710779 32373 raft_consensus.cc:348] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260807 00:06:15.711123 32373 raft_consensus.cc:374] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:15.711205 32373 raft_consensus.cc:729] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 261e70f9ff234bf5a0cfe7d3623ec132, State: Initialized, Role: FOLLOWER
I20260807 00:06:15.711741 32373 consensus_queue.cc:260] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260807 00:06:15.711927 32373 raft_consensus.cc:388] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:15.712007 32373 raft_consensus.cc:482] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:15.712127 32373 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:15.712239 32379 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:15.710458 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39001 (local address 127.0.0.1:34971)
0807 00:06:15.710588 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:15.710593 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:15.710828 (+ 235us) server_negotiation.cc:407] Connection header received
0807 00:06:15.710937 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:15.710940 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:15.710982 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:15.711060 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:15.711822 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:15.711826 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:15.711829 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:15.711842 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:15.711876 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:15.711890 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:15.711983 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:15.712067 (+ 84us) server_negotiation.cc:299] Negotiation successful
0807 00:06:15.712112 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:06:15.713303 32373 raft_consensus.cc:504] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260807 00:06:15.713670 32373 leader_election.cc:302] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 261e70f9ff234bf5a0cfe7d3623ec132; no voters:
I20260807 00:06:15.713994 32373 leader_election.cc:288] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:15.714133 32388 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:15.714366 32388 raft_consensus.cc:686] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [term 1 LEADER]: Becoming Leader. State: Replica: 261e70f9ff234bf5a0cfe7d3623ec132, State: Running, Role: LEADER
I20260807 00:06:15.714696 32388 consensus_queue.cc:237] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } }
I20260807 00:06:15.715392 32373 sys_catalog.cc:564] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:15.716610 32391 sys_catalog.cc:455] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } } }
I20260807 00:06:15.716616 32392 sys_catalog.cc:455] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 261e70f9ff234bf5a0cfe7d3623ec132. Latest consensus state: current_term: 1 leader_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "261e70f9ff234bf5a0cfe7d3623ec132" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34971 } } }
I20260807 00:06:15.716811 32391 sys_catalog.cc:458] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:15.716836 32392 sys_catalog.cc:458] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:15.717262 32402 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:15.720443 32402 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:15.726114 32402 catalog_manager.cc:1347] Generated new cluster ID: 8cd744904e824bed8df8082a398b44e5
I20260807 00:06:15.726203 32402 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:15.744903 32402 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:15.745357 32402 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:15.766395 32402 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 261e70f9ff234bf5a0cfe7d3623ec132: Generated new TSK 0
I20260807 00:06:15.767271 32402 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:15.773345 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 32267
2026-08-07T00:06:15Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:15 dist-test-slave-pww0 krb5kdc[32418](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:15 dist-test-slave-pww0 krb5kdc[32418](info): set up 2 sockets
Aug 07 00:06:15 dist-test-slave-pww0 krb5kdc[32418](info): commencing operation
krb5kdc: starting...
W20260807 00:06:16.942934 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.139s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:17 dist-test-slave-pww0 krb5kdc[32418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:17Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:17Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:17.083354 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46245
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46573
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:46245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:17.206050 32438 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:17.206375 32438 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:17.206439 32438 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:17.211246 32438 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:17.211328 32438 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:17.211355 32438 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:17.211376 32438 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:17.211397 32438 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:17.217231 32438 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46573
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:46245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:46245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32438
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:17.218475 32438 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:17.219719 32438 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:17.228969 32456 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:17.228945 32455 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:17.228969 32459 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:17.229352 32438 server_base.cc:1034] running on GCE node
I20260807 00:06:17.229966 32438 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:17.231397 32438 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:17.232599 32438 hybrid_clock.cc:648] HybridClock initialized: now 1786061177232582 us; error 31 us; skew 500 ppm
Aug 07 00:06:17 dist-test-slave-pww0 krb5kdc[32418](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061177, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:17.235605 32438 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:17.236714 32438 webserver.cc:460] Webserver started at http://127.0.0.1:44297/ using document root <none> and password file <none>
I20260807 00:06:17.237360 32438 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:17.237443 32438 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:17.237669 32438 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:17.239969 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "607ac98296904bb4bd13fb7f460cdd75"
format_stamp: "Formatted at 2026-08-07 00:06:17 on dist-test-slave-pww0"
server_key: "5646e873994146a6821125820cf15552"
server_key_iv: "cc67fca268166d4bcc4e5d0809731ac3"
server_key_version: "encryptionkey@0"
I20260807 00:06:17.240590 32438 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "607ac98296904bb4bd13fb7f460cdd75"
format_stamp: "Formatted at 2026-08-07 00:06:17 on dist-test-slave-pww0"
server_key: "5646e873994146a6821125820cf15552"
server_key_iv: "cc67fca268166d4bcc4e5d0809731ac3"
server_key_version: "encryptionkey@0"
I20260807 00:06:17.244813 32438 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.001s sys 0.003s
I20260807 00:06:17.247663 32473 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:17.249156 32438 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:17.249321 32438 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "607ac98296904bb4bd13fb7f460cdd75"
format_stamp: "Formatted at 2026-08-07 00:06:17 on dist-test-slave-pww0"
server_key: "5646e873994146a6821125820cf15552"
server_key_iv: "cc67fca268166d4bcc4e5d0809731ac3"
server_key_version: "encryptionkey@0"
I20260807 00:06:17.249449 32438 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:17.276966 32438 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:17.280277 32438 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:17.280493 32438 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:17.288206 32438 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:46245
I20260807 00:06:17.288232 32557 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:46245 every 8 connection(s)
I20260807 00:06:17.289253 32438 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:17.292147 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 32438
I20260807 00:06:17.292294 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:17.292411 32559 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:17.292624 31537 external_mini_cluster.cc:1428] Setting key 7c6cc259b36b6c8ca83b0fa826db7f78
I20260807 00:06:17.300042 32559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75: Bootstrap starting.
I20260807 00:06:17.302713 32559 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75: Neither blocks nor log segments found. Creating new log.
Aug 07 00:06:17 dist-test-slave-pww0 krb5kdc[32418](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061177, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:06:17.303610 32559 log.cc:824] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:17.306253 32559 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75: No bootstrap required, opened a new log
I20260807 00:06:17.309798 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:17.294271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:42283 (local address 127.0.0.1:46245)
0807 00:06:17.294801 (+ 530us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:17.294817 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:17.294862 (+ 45us) server_negotiation.cc:407] Connection header received
0807 00:06:17.295856 (+ 994us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:17.295884 (+ 28us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:17.296399 (+ 515us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:17.296803 (+ 404us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:17.297508 (+ 705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:17.299005 (+ 1497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:17.300050 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:17.300751 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:17.300912 (+ 161us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:17.303566 (+ 2654us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:17.303573 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:17.303592 (+ 19us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:17.303632 (+ 40us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:17.306076 (+ 2444us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:17.306770 (+ 694us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:17.306777 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:17.306783 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:17.306940 (+ 157us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:17.307343 (+ 403us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:17.307347 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:17.307348 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:17.307912 (+ 564us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:17.308137 (+ 225us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:17.308449 (+ 312us) server_negotiation.cc:299] Negotiation successful
0807 00:06:17.308725 (+ 276us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":365,"thread_start_us":165,"threads_started":1}
I20260807 00:06:17.311002 32559 raft_consensus.cc:348] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607ac98296904bb4bd13fb7f460cdd75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46245 } }
I20260807 00:06:17.311278 32559 raft_consensus.cc:374] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:17.311340 32559 raft_consensus.cc:729] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 607ac98296904bb4bd13fb7f460cdd75, State: Initialized, Role: FOLLOWER
I20260807 00:06:17.311852 32559 consensus_queue.cc:260] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607ac98296904bb4bd13fb7f460cdd75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46245 } }
I20260807 00:06:17.312028 32559 raft_consensus.cc:388] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:17.312099 32559 raft_consensus.cc:482] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:17.312201 32559 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:17.313155 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:17.313360 32559 raft_consensus.cc:504] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607ac98296904bb4bd13fb7f460cdd75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46245 } }
I20260807 00:06:17.313725 32559 leader_election.cc:302] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 607ac98296904bb4bd13fb7f460cdd75; no voters:
I20260807 00:06:17.314049 32559 leader_election.cc:288] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:17.314180 32571 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:17.314417 32571 raft_consensus.cc:686] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [term 1 LEADER]: Becoming Leader. State: Replica: 607ac98296904bb4bd13fb7f460cdd75, State: Running, Role: LEADER
I20260807 00:06:17.314774 32571 consensus_queue.cc:237] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607ac98296904bb4bd13fb7f460cdd75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46245 } }
I20260807 00:06:17.315415 32559 sys_catalog.cc:564] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:17.316112 32575 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 607ac98296904bb4bd13fb7f460cdd75. Latest consensus state: current_term: 1 leader_uuid: "607ac98296904bb4bd13fb7f460cdd75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607ac98296904bb4bd13fb7f460cdd75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46245 } } }
I20260807 00:06:17.316244 32575 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:17.316943 32572 sys_catalog.cc:455] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "607ac98296904bb4bd13fb7f460cdd75" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "607ac98296904bb4bd13fb7f460cdd75" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 46245 } } }
I20260807 00:06:17.317054 32572 sys_catalog.cc:458] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75 [sys.catalog]: This master's current role is: LEADER
W20260807 00:06:17.320979 32603 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:17.321072 32603 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:17.321143 32591 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:17.321978 32591 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:17.325678 32567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:17.315405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:37249 (local address 127.0.0.1:46245)
0807 00:06:17.315538 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:17.315543 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:17.315972 (+ 429us) server_negotiation.cc:407] Connection header received
0807 00:06:17.316195 (+ 223us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:17.316198 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:17.316253 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:17.316308 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:17.317507 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:17.318253 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:17.319401 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:17.319765 (+ 364us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:17.319884 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:17.321676 (+ 1792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:17.321680 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:17.321683 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:17.321715 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:17.323589 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:17.324166 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:17.324170 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:17.324173 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:17.324279 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:17.324651 (+ 372us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:17.324655 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:17.324658 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:17.324925 (+ 267us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:17.325095 (+ 170us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:17.325380 (+ 285us) server_negotiation.cc:299] Negotiation successful
0807 00:06:17.325496 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260807 00:06:17.328110 32591 catalog_manager.cc:1347] Generated new cluster ID: 58823ceeffff4794b3873c0241256900
I20260807 00:06:17.328193 32591 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:17.338346 32591 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:17.339578 32591 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:17.351315 32591 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 607ac98296904bb4bd13fb7f460cdd75: Generated new TSK 0
I20260807 00:06:17.352342 32591 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:17.360522 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 32438
W20260807 00:06:17.371338 32416 connection.cc:535] client connection to 127.0.0.1:46245 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:46245): Connection reset by peer (error 104)
2026-08-07T00:06:17Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1582 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:17 dist-test-slave-pww0 krb5kdc[32617](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:17 dist-test-slave-pww0 krb5kdc[32617](info): set up 2 sockets
Aug 07 00:06:17 dist-test-slave-pww0 krb5kdc[32617](info): commencing operation
krb5kdc: starting...
W20260807 00:06:18.520591 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:18 dist-test-slave-pww0 krb5kdc[32617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061178, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:18.656591 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42325
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46869
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42325
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:18.776304 32644 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:18.776615 32644 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:18.776680 32644 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:18.780845 32644 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:18.780926 32644 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:18.780954 32644 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:18.780977 32644 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:18.781003 32644 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:18.786760 32644 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46869
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42325
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42325
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32644
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:18.788046 32644 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:18.789176 32644 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:18.798384 32653 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:18.798333 32660 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:18.798483 32655 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:18.798882 32644 server_base.cc:1034] running on GCE node
I20260807 00:06:18.799521 32644 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:18.800704 32644 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:18.801891 32644 hybrid_clock.cc:648] HybridClock initialized: now 1786061178801875 us; error 37 us; skew 500 ppm
Aug 07 00:06:18 dist-test-slave-pww0 krb5kdc[32617](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061178, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:18.804811 32644 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:18.805817 32644 webserver.cc:460] Webserver started at http://127.0.0.1:43511/ using document root <none> and password file <none>
I20260807 00:06:18.806393 32644 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:18.806466 32644 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:18.806689 32644 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:18.808976 32644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "348567e51efe4db0acf678c85eaa0dfc"
format_stamp: "Formatted at 2026-08-07 00:06:18 on dist-test-slave-pww0"
server_key: "5356ea29f937201bed14c41805c2ce99"
server_key_iv: "67f9ca2739c30b32f77a73240ff64a03"
server_key_version: "encryptionkey@0"
I20260807 00:06:18.809585 32644 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "348567e51efe4db0acf678c85eaa0dfc"
format_stamp: "Formatted at 2026-08-07 00:06:18 on dist-test-slave-pww0"
server_key: "5356ea29f937201bed14c41805c2ce99"
server_key_iv: "67f9ca2739c30b32f77a73240ff64a03"
server_key_version: "encryptionkey@0"
I20260807 00:06:18.813845 32644 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:06:18.816507 32673 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:18.817797 32644 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.003s sys 0.000s
I20260807 00:06:18.817927 32644 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "348567e51efe4db0acf678c85eaa0dfc"
format_stamp: "Formatted at 2026-08-07 00:06:18 on dist-test-slave-pww0"
server_key: "5356ea29f937201bed14c41805c2ce99"
server_key_iv: "67f9ca2739c30b32f77a73240ff64a03"
server_key_version: "encryptionkey@0"
I20260807 00:06:18.818034 32644 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:18.836357 32644 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:18.839901 32644 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:18.840108 32644 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:18.847599 32644 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42325
I20260807 00:06:18.847675 32760 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42325 every 8 connection(s)
I20260807 00:06:18.848680 32644 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:18.851951 32761 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:18.854995 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 32644
I20260807 00:06:18.855161 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:18.855444 31537 external_mini_cluster.cc:1428] Setting key 797cc003d31d0a31c73eee322fe8e4b3
I20260807 00:06:18.858410 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc: Bootstrap starting.
I20260807 00:06:18.860991 32761 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:18.861990 32761 log.cc:824] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:18.864496 32761 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc: No bootstrap required, opened a new log
I20260807 00:06:18.865811 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:18.857131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:60877 (local address 127.0.0.1:42325)
0807 00:06:18.857663 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:18.857682 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:18.857723 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:06:18.858422 (+ 699us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:18.858442 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:18.858865 (+ 423us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:18.859181 (+ 316us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:18.859598 (+ 417us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:18.860641 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:18.861919 (+ 1278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:18.862479 (+ 560us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:18.862620 (+ 141us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:18.863600 (+ 980us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:18.863609 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:18.863629 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:18.864161 (+ 532us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:18.864241 (+ 80us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:18.864275 (+ 34us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:18.864413 (+ 138us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:18.864642 (+ 229us) server_negotiation.cc:299] Negotiation successful
0807 00:06:18.864796 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":367,"thread_start_us":172,"threads_started":1}
I20260807 00:06:18.869277 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:18.869112 32761 raft_consensus.cc:348] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348567e51efe4db0acf678c85eaa0dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42325 } }
I20260807 00:06:18.869393 32761 raft_consensus.cc:374] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:18.869454 32761 raft_consensus.cc:729] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 348567e51efe4db0acf678c85eaa0dfc, State: Initialized, Role: FOLLOWER
I20260807 00:06:18.869911 32761 consensus_queue.cc:260] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348567e51efe4db0acf678c85eaa0dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42325 } }
I20260807 00:06:18.870052 32761 raft_consensus.cc:388] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:18.870119 32761 raft_consensus.cc:482] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:18.870220 32761 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:18.871332 32761 raft_consensus.cc:504] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348567e51efe4db0acf678c85eaa0dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42325 } }
I20260807 00:06:18.871764 32761 leader_election.cc:302] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 348567e51efe4db0acf678c85eaa0dfc; no voters:
I20260807 00:06:18.872128 32761 leader_election.cc:288] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:18.872223 311 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:18.872471 311 raft_consensus.cc:686] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [term 1 LEADER]: Becoming Leader. State: Replica: 348567e51efe4db0acf678c85eaa0dfc, State: Running, Role: LEADER
I20260807 00:06:18.872870 311 consensus_queue.cc:237] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348567e51efe4db0acf678c85eaa0dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42325 } }
I20260807 00:06:18.873595 32761 sys_catalog.cc:564] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:18.874709 314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "348567e51efe4db0acf678c85eaa0dfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348567e51efe4db0acf678c85eaa0dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42325 } } }
I20260807 00:06:18.874922 314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:18.875217 314 sys_catalog.cc:455] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 348567e51efe4db0acf678c85eaa0dfc. Latest consensus state: current_term: 1 leader_uuid: "348567e51efe4db0acf678c85eaa0dfc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "348567e51efe4db0acf678c85eaa0dfc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42325 } } }
I20260807 00:06:18.875385 314 sys_catalog.cc:458] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:18.875764 331 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:18.876480 301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:18.871642 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:46383 (local address 127.0.0.1:42325)
0807 00:06:18.871784 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:18.871789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:18.872003 (+ 214us) server_negotiation.cc:407] Connection header received
0807 00:06:18.872155 (+ 152us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:18.872158 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:18.872214 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:18.872283 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:18.872595 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:18.873198 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:18.874127 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:18.874465 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:18.874542 (+ 77us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:18.875772 (+ 1230us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:18.875776 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:18.875779 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:18.875793 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:18.875824 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:18.875838 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:18.875899 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:18.876295 (+ 396us) server_negotiation.cc:299] Negotiation successful
0807 00:06:18.876343 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:06:18.879683 331 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:18.885462 331 catalog_manager.cc:1347] Generated new cluster ID: dd5e25a3dd0e4e87b418fe189e1a9c82
I20260807 00:06:18.885557 331 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:18.933054 331 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:18.934051 331 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:18.952728 331 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 348567e51efe4db0acf678c85eaa0dfc: Generated new TSK 0
I20260807 00:06:18.953574 331 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:18.961794 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 32644
W20260807 00:06:18.985167 32612 connection.cc:535] client connection to 127.0.0.1:42325 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:42325): Connection reset by peer (error 104)
2026-08-07T00:06:18Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1613 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:19 dist-test-slave-pww0 krb5kdc[347](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:19 dist-test-slave-pww0 krb5kdc[347](info): set up 2 sockets
Aug 07 00:06:19 dist-test-slave-pww0 krb5kdc[347](info): commencing operation
krb5kdc: starting...
W20260807 00:06:20.134105 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.003s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:20 dist-test-slave-pww0 krb5kdc[347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061180, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:20Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:20Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:20.271680 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39513
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37729
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39513
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:20.393008 370 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:20.393340 370 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:20.393404 370 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:20.397675 370 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:20.397765 370 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:20.397794 370 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:20.397816 370 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:20.397836 370 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:20.403640 370 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37729
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39513
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39513
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.370
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:20.404898 370 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:20.406029 370 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:06:20.415223 370 server_base.cc:1034] running on GCE node
W20260807 00:06:20.415179 375 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:20.415196 381 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:20.415179 376 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:20.416039 370 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:20.417268 370 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:20.418466 370 hybrid_clock.cc:648] HybridClock initialized: now 1786061180418431 us; error 49 us; skew 500 ppm
Aug 07 00:06:20 dist-test-slave-pww0 krb5kdc[347](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061180, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:20.426004 370 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:20.427179 370 webserver.cc:460] Webserver started at http://127.0.0.1:44007/ using document root <none> and password file <none>
I20260807 00:06:20.427857 370 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:20.427938 370 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:20.428164 370 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:20.430439 370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1"
format_stamp: "Formatted at 2026-08-07 00:06:20 on dist-test-slave-pww0"
server_key: "9e6a7491d22834536fc3c9e3aa38cfa0"
server_key_iv: "543b668a939d10a9fbbefd72f11ba79c"
server_key_version: "encryptionkey@0"
I20260807 00:06:20.431034 370 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1"
format_stamp: "Formatted at 2026-08-07 00:06:20 on dist-test-slave-pww0"
server_key: "9e6a7491d22834536fc3c9e3aa38cfa0"
server_key_iv: "543b668a939d10a9fbbefd72f11ba79c"
server_key_version: "encryptionkey@0"
I20260807 00:06:20.435247 370 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.007s sys 0.000s
I20260807 00:06:20.438259 397 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:20.439749 370 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:20.439910 370 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1"
format_stamp: "Formatted at 2026-08-07 00:06:20 on dist-test-slave-pww0"
server_key: "9e6a7491d22834536fc3c9e3aa38cfa0"
server_key_iv: "543b668a939d10a9fbbefd72f11ba79c"
server_key_version: "encryptionkey@0"
I20260807 00:06:20.440040 370 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:20.445205 370 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:20.448637 370 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:20.448832 370 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:20.456568 370 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39513
I20260807 00:06:20.456612 482 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39513 every 8 connection(s)
I20260807 00:06:20.457620 370 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:20.459983 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 370
I20260807 00:06:20.460119 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:20.460451 31537 external_mini_cluster.cc:1428] Setting key b4405ebbf8021e7945e9e3c98012e58a
I20260807 00:06:20.460925 485 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:20.466014 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:20.461981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:46791 (local address 127.0.0.1:39513)
0807 00:06:20.462476 (+ 495us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:20.462492 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:20.462529 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:06:20.463219 (+ 690us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:20.463239 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:20.463584 (+ 345us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:20.463923 (+ 339us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:20.464759 (+ 836us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:20.464763 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:20.464776 (+ 13us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:20.464787 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:20.464849 (+ 62us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:20.464870 (+ 21us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:20.465006 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:20.465081 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 00:06:20.465254 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":167,"threads_started":1}
I20260807 00:06:20.468163 485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1: Bootstrap starting.
I20260807 00:06:20.469197 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:20.471271 485 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:20.472311 485 log.cc:824] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:20.473305 488 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:20.471677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:53045 (local address 127.0.0.1:39513)
0807 00:06:20.471817 (+ 140us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:20.471822 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:20.471988 (+ 166us) server_negotiation.cc:407] Connection header received
0807 00:06:20.472136 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:20.472138 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:20.472173 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:20.472235 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:20.472929 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:20.472933 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:20.472935 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:20.472947 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:20.472973 (+ 26us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:20.472986 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:20.473077 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:20.473137 (+ 60us) server_negotiation.cc:299] Negotiation successful
0807 00:06:20.473179 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:06:20.474906 485 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1: No bootstrap required, opened a new log
I20260807 00:06:20.478708 485 raft_consensus.cc:348] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39513 } }
I20260807 00:06:20.478993 485 raft_consensus.cc:374] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:20.479059 485 raft_consensus.cc:729] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2b42eef8a6b40fab1cccf1fb87b94a1, State: Initialized, Role: FOLLOWER
I20260807 00:06:20.479569 485 consensus_queue.cc:260] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39513 } }
I20260807 00:06:20.479724 485 raft_consensus.cc:388] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:20.479795 485 raft_consensus.cc:482] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:20.479902 485 raft_consensus.cc:3037] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:20.480929 485 raft_consensus.cc:504] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39513 } }
I20260807 00:06:20.481312 485 leader_election.cc:302] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: f2b42eef8a6b40fab1cccf1fb87b94a1; no voters:
I20260807 00:06:20.481657 485 leader_election.cc:288] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:20.481814 500 raft_consensus.cc:2781] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:20.482054 500 raft_consensus.cc:686] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [term 1 LEADER]: Becoming Leader. State: Replica: f2b42eef8a6b40fab1cccf1fb87b94a1, State: Running, Role: LEADER
I20260807 00:06:20.482376 500 consensus_queue.cc:237] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39513 } }
I20260807 00:06:20.483012 485 sys_catalog.cc:564] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:20.484381 501 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39513 } } }
I20260807 00:06:20.484511 501 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:20.484854 503 sys_catalog.cc:455] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader f2b42eef8a6b40fab1cccf1fb87b94a1. Latest consensus state: current_term: 1 leader_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "f2b42eef8a6b40fab1cccf1fb87b94a1" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39513 } } }
I20260807 00:06:20.484977 503 sys_catalog.cc:458] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:20.485018 514 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:20.488026 514 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:20.493891 514 catalog_manager.cc:1347] Generated new cluster ID: 5e4d5ad05d474108ac7e0309b14ad4ba
I20260807 00:06:20.493990 514 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:20.512014 514 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:20.512393 514 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:20.521023 514 catalog_manager.cc:5868] T 00000000000000000000000000000000 P f2b42eef8a6b40fab1cccf1fb87b94a1: Generated new TSK 0
I20260807 00:06:20.521824 514 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:20.533821 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 370
2026-08-07T00:06:20Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1564 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:20 dist-test-slave-pww0 krb5kdc[535](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:20 dist-test-slave-pww0 krb5kdc[535](info): set up 2 sockets
Aug 07 00:06:20 dist-test-slave-pww0 krb5kdc[535](info): commencing operation
krb5kdc: starting...
W20260807 00:06:21.712706 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.006s sys 0.002s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:21 dist-test-slave-pww0 krb5kdc[535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:21Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:21Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:21.844309 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37409
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40051
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37409
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:21.958906 558 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:21.959199 558 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:21.959249 558 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:21.963340 558 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:21.963411 558 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:21.963430 558 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:21.963446 558 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:21.963512 558 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:21.969007 558 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40051
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37409
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37409
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.558
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:21.970161 558 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:21.971284 558 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:21.980379 566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:21.980535 569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:21.980423 567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:21.980417 558 server_base.cc:1034] running on GCE node
I20260807 00:06:21.981246 558 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:21.982508 558 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:21.983675 558 hybrid_clock.cc:648] HybridClock initialized: now 1786061181983659 us; error 38 us; skew 500 ppm
Aug 07 00:06:21 dist-test-slave-pww0 krb5kdc[535](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061181, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:21.986613 558 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:21.987684 558 webserver.cc:460] Webserver started at http://127.0.0.1:46655/ using document root <none> and password file <none>
I20260807 00:06:21.988266 558 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:21.988329 558 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:21.988511 558 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:21.990744 558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "65682feded164514a955df9e010de598"
format_stamp: "Formatted at 2026-08-07 00:06:21 on dist-test-slave-pww0"
server_key: "8988bd70c0713453ca97e4dfcf1d28d8"
server_key_iv: "7fa76046312e467d6ad9e09ee20749a4"
server_key_version: "encryptionkey@0"
I20260807 00:06:21.991364 558 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "65682feded164514a955df9e010de598"
format_stamp: "Formatted at 2026-08-07 00:06:21 on dist-test-slave-pww0"
server_key: "8988bd70c0713453ca97e4dfcf1d28d8"
server_key_iv: "7fa76046312e467d6ad9e09ee20749a4"
server_key_version: "encryptionkey@0"
I20260807 00:06:21.995515 558 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.000s
I20260807 00:06:21.998337 579 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:21.999835 558 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:21.999965 558 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "65682feded164514a955df9e010de598"
format_stamp: "Formatted at 2026-08-07 00:06:21 on dist-test-slave-pww0"
server_key: "8988bd70c0713453ca97e4dfcf1d28d8"
server_key_iv: "7fa76046312e467d6ad9e09ee20749a4"
server_key_version: "encryptionkey@0"
I20260807 00:06:22.000090 558 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:22.018414 558 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:22.021915 558 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:22.022113 558 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:22.029739 558 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37409
I20260807 00:06:22.029770 670 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37409 every 8 connection(s)
I20260807 00:06:22.030839 558 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:22.032166 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 558
I20260807 00:06:22.032307 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:22.032646 31537 external_mini_cluster.cc:1428] Setting key a3a2975aea5b1e79e0bdcef5e53702f2
I20260807 00:06:22.034395 673 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:22.040433 673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598: Bootstrap starting.
Aug 07 00:06:22 dist-test-slave-pww0 krb5kdc[535](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061181, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260807 00:06:22.043157 673 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:22.044030 673 log.cc:824] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:22.046479 673 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598: No bootstrap required, opened a new log
I20260807 00:06:22.047418 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:22.034150 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:42405 (local address 127.0.0.1:37409)
0807 00:06:22.034640 (+ 490us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:22.034658 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:22.034696 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:06:22.035400 (+ 704us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:22.035419 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:22.035831 (+ 412us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:22.036154 (+ 323us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:22.036965 (+ 811us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:22.037988 (+ 1023us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:22.038853 (+ 865us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:22.039336 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:22.039530 (+ 194us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:22.042442 (+ 2912us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:22.042448 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:22.042462 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:22.042497 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:22.044322 (+ 1825us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:22.044884 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:22.044890 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:22.044895 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:22.045016 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:22.045364 (+ 348us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:22.045366 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:22.045367 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:22.045776 (+ 409us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:22.045937 (+ 161us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:22.046338 (+ 401us) server_negotiation.cc:299] Negotiation successful
0807 00:06:22.046600 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":167,"threads_started":1}
I20260807 00:06:22.050840 673 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65682feded164514a955df9e010de598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37409 } }
I20260807 00:06:22.051168 673 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:22.051265 673 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65682feded164514a955df9e010de598, State: Initialized, Role: FOLLOWER
I20260807 00:06:22.051543 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:22.051795 673 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65682feded164514a955df9e010de598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37409 } }
I20260807 00:06:22.051956 673 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:22.052024 673 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:22.052157 673 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:22.053529 673 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65682feded164514a955df9e010de598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37409 } }
I20260807 00:06:22.054045 673 leader_election.cc:302] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65682feded164514a955df9e010de598; no voters:
I20260807 00:06:22.054492 673 leader_election.cc:288] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:22.054589 692 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:22.054785 692 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [term 1 LEADER]: Becoming Leader. State: Replica: 65682feded164514a955df9e010de598, State: Running, Role: LEADER
I20260807 00:06:22.055135 692 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65682feded164514a955df9e010de598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37409 } }
I20260807 00:06:22.056043 673 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:22.057700 692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65682feded164514a955df9e010de598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65682feded164514a955df9e010de598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37409 } } }
I20260807 00:06:22.057938 692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:22.058185 692 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65682feded164514a955df9e010de598. Latest consensus state: current_term: 1 leader_uuid: "65682feded164514a955df9e010de598" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65682feded164514a955df9e010de598" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37409 } } }
I20260807 00:06:22.058327 692 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:22.058436 703 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:22.061961 703 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:22.070189 680 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:22.054048 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:51183 (local address 127.0.0.1:37409)
0807 00:06:22.054193 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:22.054197 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:22.059882 (+ 5685us) server_negotiation.cc:407] Connection header received
0807 00:06:22.060033 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:22.060037 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:22.060097 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:22.060178 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:22.062660 (+ 2482us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:22.063396 (+ 736us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:22.064420 (+ 1024us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:22.064744 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:22.064853 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:22.066727 (+ 1874us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:22.066731 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:22.066734 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:22.066764 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:22.068156 (+ 1392us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:22.068757 (+ 601us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:22.068762 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:22.068763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:22.068864 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:22.069241 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:22.069245 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:22.069247 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:22.069476 (+ 229us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:22.069621 (+ 145us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:22.069935 (+ 314us) server_negotiation.cc:299] Negotiation successful
0807 00:06:22.070028 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:06:22.070259 703 catalog_manager.cc:1347] Generated new cluster ID: 2ad7e07105774c4ba35aa9a1b2c18634
I20260807 00:06:22.070335 703 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:22.078069 703 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:22.079068 703 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:22.089952 703 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65682feded164514a955df9e010de598: Generated new TSK 0
I20260807 00:06:22.090721 703 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:22.097900 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 558
W20260807 00:06:22.116400 532 connection.cc:535] client connection to 127.0.0.1:37409 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:37409): Connection reset by peer (error 104)
2026-08-07T00:06:22Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1567 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:22 dist-test-slave-pww0 krb5kdc[724](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:22 dist-test-slave-pww0 krb5kdc[724](info): set up 2 sockets
Aug 07 00:06:22 dist-test-slave-pww0 krb5kdc[724](info): commencing operation
krb5kdc: starting...
W20260807 00:06:23.281910 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.151s user 0.000s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:23 dist-test-slave-pww0 krb5kdc[724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061183, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:23Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:23.418396 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40445
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:33929
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40445
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:23.539193 750 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:23.539543 750 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:23.539613 750 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:23.543797 750 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:23.543879 750 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:23.543906 750 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:23.543928 750 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:23.543951 750 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:23.549700 750 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:33929
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40445
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40445
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.750
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:23.550974 750 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:23.552174 750 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:23.561448 758 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:23.561450 756 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:23.561527 750 server_base.cc:1034] running on GCE node
W20260807 00:06:23.561450 764 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:23.562361 750 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:23.563650 750 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:23.564838 750 hybrid_clock.cc:648] HybridClock initialized: now 1786061183564817 us; error 37 us; skew 500 ppm
Aug 07 00:06:23 dist-test-slave-pww0 krb5kdc[724](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061183, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:23.567905 750 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:23.568953 750 webserver.cc:460] Webserver started at http://127.0.0.1:39839/ using document root <none> and password file <none>
I20260807 00:06:23.569550 750 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:23.569626 750 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:23.569854 750 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:23.572263 750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "12d43261551040d28cc6b018a08fc9b3"
format_stamp: "Formatted at 2026-08-07 00:06:23 on dist-test-slave-pww0"
server_key: "b232e45e0658438d94237219f08b0d1a"
server_key_iv: "ab2e82ab08612709b678dec43f903ffb"
server_key_version: "encryptionkey@0"
I20260807 00:06:23.572903 750 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "12d43261551040d28cc6b018a08fc9b3"
format_stamp: "Formatted at 2026-08-07 00:06:23 on dist-test-slave-pww0"
server_key: "b232e45e0658438d94237219f08b0d1a"
server_key_iv: "ab2e82ab08612709b678dec43f903ffb"
server_key_version: "encryptionkey@0"
I20260807 00:06:23.577188 750 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:06:23.580240 773 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:23.581682 750 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.004s sys 0.000s
I20260807 00:06:23.581856 750 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "12d43261551040d28cc6b018a08fc9b3"
format_stamp: "Formatted at 2026-08-07 00:06:23 on dist-test-slave-pww0"
server_key: "b232e45e0658438d94237219f08b0d1a"
server_key_iv: "ab2e82ab08612709b678dec43f903ffb"
server_key_version: "encryptionkey@0"
I20260807 00:06:23.581982 750 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:23.595012 750 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:23.598502 750 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:23.598814 750 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:23.605969 750 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40445
I20260807 00:06:23.606016 865 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40445 every 8 connection(s)
I20260807 00:06:23.607010 750 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:23.610109 867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:23.616045 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3: Bootstrap starting.
I20260807 00:06:23.616751 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 750
I20260807 00:06:23.616868 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:23.617133 31537 external_mini_cluster.cc:1428] Setting key 9818ce742c7269a7be095833daa12730
I20260807 00:06:23.619035 867 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:23.619894 867 log.cc:824] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:23.622275 867 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3: No bootstrap required, opened a new log
I20260807 00:06:23.626693 867 raft_consensus.cc:348] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d43261551040d28cc6b018a08fc9b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40445 } }
I20260807 00:06:23.626979 867 raft_consensus.cc:374] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:23.627039 867 raft_consensus.cc:729] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12d43261551040d28cc6b018a08fc9b3, State: Initialized, Role: FOLLOWER
I20260807 00:06:23.627240 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:23.618455 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:35591 (local address 127.0.0.1:40445)
0807 00:06:23.618941 (+ 486us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:23.618958 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:23.618996 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:06:23.619822 (+ 826us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:23.619843 (+ 21us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:23.620238 (+ 395us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:23.620539 (+ 301us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:23.620960 (+ 421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:23.622376 (+ 1416us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:23.623348 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:23.624090 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:23.624261 (+ 171us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:23.625116 (+ 855us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:23.625123 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:23.625145 (+ 22us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:23.625699 (+ 554us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:23.625787 (+ 88us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:23.625820 (+ 33us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:23.625957 (+ 137us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:23.626068 (+ 111us) server_negotiation.cc:299] Negotiation successful
0807 00:06:23.626217 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":335,"thread_start_us":133,"threads_started":1}
I20260807 00:06:23.627574 867 consensus_queue.cc:260] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d43261551040d28cc6b018a08fc9b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40445 } }
I20260807 00:06:23.627712 867 raft_consensus.cc:388] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:23.627774 867 raft_consensus.cc:482] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:23.627873 867 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:23.628801 867 raft_consensus.cc:504] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d43261551040d28cc6b018a08fc9b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40445 } }
I20260807 00:06:23.629127 867 leader_election.cc:302] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 12d43261551040d28cc6b018a08fc9b3; no voters:
I20260807 00:06:23.629499 867 leader_election.cc:288] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:23.629626 875 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:23.629868 875 raft_consensus.cc:686] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [term 1 LEADER]: Becoming Leader. State: Replica: 12d43261551040d28cc6b018a08fc9b3, State: Running, Role: LEADER
I20260807 00:06:23.630293 875 consensus_queue.cc:237] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d43261551040d28cc6b018a08fc9b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40445 } }
I20260807 00:06:23.630370 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:23.630844 867 sys_catalog.cc:564] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:23.631826 876 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 12d43261551040d28cc6b018a08fc9b3. Latest consensus state: current_term: 1 leader_uuid: "12d43261551040d28cc6b018a08fc9b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d43261551040d28cc6b018a08fc9b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40445 } } }
I20260807 00:06:23.631958 876 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:23.632575 877 sys_catalog.cc:455] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "12d43261551040d28cc6b018a08fc9b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "12d43261551040d28cc6b018a08fc9b3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40445 } } }
I20260807 00:06:23.632668 877 sys_catalog.cc:458] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:23.633478 889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:23.636932 889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:23.637457 873 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:23.632735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:38283 (local address 127.0.0.1:40445)
0807 00:06:23.632909 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:23.632913 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:23.633122 (+ 209us) server_negotiation.cc:407] Connection header received
0807 00:06:23.633262 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:23.633265 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:23.633319 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:23.633393 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:23.633818 (+ 425us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:23.634563 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:23.635768 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:23.635993 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:23.636084 (+ 91us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:23.636956 (+ 872us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:23.636959 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:23.636962 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:23.636975 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:23.637006 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:23.637019 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:23.637101 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:23.637278 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 00:06:23.637322 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":91}
I20260807 00:06:23.643417 889 catalog_manager.cc:1347] Generated new cluster ID: 55e09eb5193647feb16e0fd7b3424ab4
I20260807 00:06:23.643518 889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:23.653133 889 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:23.654130 889 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:23.665156 889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 12d43261551040d28cc6b018a08fc9b3: Generated new TSK 0
I20260807 00:06:23.666126 889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:23.676810 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 750
W20260807 00:06:23.694636 721 connection.cc:535] client connection to 127.0.0.1:40445 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40445): Connection reset by peer (error 104)
2026-08-07T00:06:23Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1578 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:23 dist-test-slave-pww0 krb5kdc[914](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:23 dist-test-slave-pww0 krb5kdc[914](info): set up 2 sockets
Aug 07 00:06:23 dist-test-slave-pww0 krb5kdc[914](info): commencing operation
krb5kdc: starting...
W20260807 00:06:24.849473 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.140s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:24 dist-test-slave-pww0 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061184, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:24Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:24Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260807 00:06:24.987001 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45041
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:46865
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45041
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:25.105891 936 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:25.106216 936 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:25.106261 936 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:25.110483 936 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:25.110564 936 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:25.110584 936 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:25.110599 936 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:25.110618 936 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:25.116394 936 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:46865
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45041
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45041
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.936
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:25.117607 936 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:25.118752 936 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:25.128775 946 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:25.132906 944 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:25.133927 942 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:25.134137 936 server_base.cc:1034] running on GCE node
I20260807 00:06:25.135097 936 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:25.136953 936 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:25.138178 936 hybrid_clock.cc:648] HybridClock initialized: now 1786061185138138 us; error 65 us; skew 500 ppm
Aug 07 00:06:25 dist-test-slave-pww0 krb5kdc[914](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061185, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:25.141775 936 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260807 00:06:25.142962 936 webserver.cc:460] Webserver started at http://127.0.0.1:33401/ using document root <none> and password file <none>
I20260807 00:06:25.143653 936 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:25.143743 936 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:25.143965 936 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:25.146443 936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "928433a2967a4cff93f4df7ceaa65ad7"
format_stamp: "Formatted at 2026-08-07 00:06:25 on dist-test-slave-pww0"
server_key: "b97896c9c4d73496ff39ad35789e2dac"
server_key_iv: "3913b6549e34cdc3ab69a60cc070b222"
server_key_version: "encryptionkey@0"
I20260807 00:06:25.147109 936 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "928433a2967a4cff93f4df7ceaa65ad7"
format_stamp: "Formatted at 2026-08-07 00:06:25 on dist-test-slave-pww0"
server_key: "b97896c9c4d73496ff39ad35789e2dac"
server_key_iv: "3913b6549e34cdc3ab69a60cc070b222"
server_key_version: "encryptionkey@0"
I20260807 00:06:25.151650 936 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.003s
I20260807 00:06:25.154800 955 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:25.156525 936 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:25.156874 936 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "928433a2967a4cff93f4df7ceaa65ad7"
format_stamp: "Formatted at 2026-08-07 00:06:25 on dist-test-slave-pww0"
server_key: "b97896c9c4d73496ff39ad35789e2dac"
server_key_iv: "3913b6549e34cdc3ab69a60cc070b222"
server_key_version: "encryptionkey@0"
I20260807 00:06:25.157040 936 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:25.162410 936 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:25.166181 936 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:25.166411 936 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:25.174844 936 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45041
I20260807 00:06:25.174894 1042 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45041 every 8 connection(s)
I20260807 00:06:25.176117 936 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:25.179625 1046 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:25.185523 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 936
I20260807 00:06:25.185659 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:25.185946 31537 external_mini_cluster.cc:1428] Setting key 9352bce3eefd1ebcd513871f52b40786
I20260807 00:06:25.186877 1046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7: Bootstrap starting.
I20260807 00:06:25.189827 1046 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:25.190872 1046 log.cc:824] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:25.192345 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:25.187636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:38875 (local address 127.0.0.1:45041)
0807 00:06:25.188220 (+ 584us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:25.188246 (+ 26us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:25.188292 (+ 46us) server_negotiation.cc:407] Connection header received
0807 00:06:25.189066 (+ 774us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:25.189090 (+ 24us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:25.189446 (+ 356us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:25.189866 (+ 420us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:25.190770 (+ 904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:25.190775 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:25.190790 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:25.190802 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:25.190865 (+ 63us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:25.190888 (+ 23us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:25.191008 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:25.191157 (+ 149us) server_negotiation.cc:299] Negotiation successful
0807 00:06:25.191360 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":390,"thread_start_us":172,"threads_started":1}
I20260807 00:06:25.193416 1046 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7: No bootstrap required, opened a new log
I20260807 00:06:25.196035 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:25.198028 1046 raft_consensus.cc:348] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928433a2967a4cff93f4df7ceaa65ad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45041 } }
I20260807 00:06:25.198333 1046 raft_consensus.cc:374] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:25.198415 1046 raft_consensus.cc:729] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 928433a2967a4cff93f4df7ceaa65ad7, State: Initialized, Role: FOLLOWER
I20260807 00:06:25.198973 1046 consensus_queue.cc:260] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928433a2967a4cff93f4df7ceaa65ad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45041 } }
I20260807 00:06:25.199132 1046 raft_consensus.cc:388] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:25.199208 1046 raft_consensus.cc:482] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:25.199323 1046 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:25.200359 1053 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:25.198407 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.195:59221 (local address 127.0.0.1:45041)
0807 00:06:25.198557 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:25.198562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:25.198786 (+ 224us) server_negotiation.cc:407] Connection header received
0807 00:06:25.198920 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:25.198923 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:25.198965 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:25.199042 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:25.199966 (+ 924us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:25.199971 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:25.199973 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:25.199987 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:25.200022 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:25.200035 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:25.200138 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:25.200174 (+ 36us) server_negotiation.cc:299] Negotiation successful
0807 00:06:25.200218 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:06:25.200685 1046 raft_consensus.cc:504] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928433a2967a4cff93f4df7ceaa65ad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45041 } }
I20260807 00:06:25.201114 1046 leader_election.cc:302] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 928433a2967a4cff93f4df7ceaa65ad7; no voters:
I20260807 00:06:25.201524 1046 leader_election.cc:288] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:25.201635 1063 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:25.201933 1063 raft_consensus.cc:686] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [term 1 LEADER]: Becoming Leader. State: Replica: 928433a2967a4cff93f4df7ceaa65ad7, State: Running, Role: LEADER
I20260807 00:06:25.202306 1063 consensus_queue.cc:237] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928433a2967a4cff93f4df7ceaa65ad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45041 } }
I20260807 00:06:25.202875 1046 sys_catalog.cc:564] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:25.204124 1066 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "928433a2967a4cff93f4df7ceaa65ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928433a2967a4cff93f4df7ceaa65ad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45041 } } }
I20260807 00:06:25.204293 1066 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:25.204648 1068 sys_catalog.cc:455] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 928433a2967a4cff93f4df7ceaa65ad7. Latest consensus state: current_term: 1 leader_uuid: "928433a2967a4cff93f4df7ceaa65ad7" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "928433a2967a4cff93f4df7ceaa65ad7" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45041 } } }
I20260807 00:06:25.204785 1068 sys_catalog.cc:458] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:25.204756 1083 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:25.208771 1083 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:25.216077 1083 catalog_manager.cc:1347] Generated new cluster ID: cbe52ccd34664f339480f6082e39fb0c
I20260807 00:06:25.216180 1083 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:25.233971 1083 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:25.234553 1083 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:25.250543 1083 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 928433a2967a4cff93f4df7ceaa65ad7: Generated new TSK 0
I20260807 00:06:25.251411 1083 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:25.263995 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 936
2026-08-07T00:06:25Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1586 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:25 dist-test-slave-pww0 krb5kdc[1102](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:25 dist-test-slave-pww0 krb5kdc[1102](info): set up 2 sockets
Aug 07 00:06:25 dist-test-slave-pww0 krb5kdc[1102](info): commencing operation
krb5kdc: starting...
W20260807 00:06:26.430187 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.134s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:26 dist-test-slave-pww0 krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:26Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:26Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:26.567241 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:42179
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41349
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:42179
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:26.685491 1129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:26.685789 1129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:26.685833 1129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:26.689935 1129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:26.690016 1129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:26.690035 1129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:26.690052 1129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:26.690066 1129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:26.696118 1129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41349
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:42179
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:42179
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:26.697258 1129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:26.698350 1129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:26.707832 1136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:26.707948 1139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:26.707854 1129 server_base.cc:1034] running on GCE node
W20260807 00:06:26.707813 1137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:26.708734 1129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:26.709975 1129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:26.711174 1129 hybrid_clock.cc:648] HybridClock initialized: now 1786061186711158 us; error 41 us; skew 500 ppm
Aug 07 00:06:26 dist-test-slave-pww0 krb5kdc[1102](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061186, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:26.718981 1129 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:26.720106 1129 webserver.cc:460] Webserver started at http://127.30.204.126:46389/ using document root <none> and password file <none>
I20260807 00:06:26.720710 1129 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:26.720783 1129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:26.721019 1129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:26.723284 1129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "b19f4233d7d7453eb60a28900a24155d"
format_stamp: "Formatted at 2026-08-07 00:06:26 on dist-test-slave-pww0"
server_key: "d4a0167283cb668d8b12169f9d863293"
server_key_iv: "556bd6a4bb4560c336f619378b90a15d"
server_key_version: "encryptionkey@0"
I20260807 00:06:26.723940 1129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "b19f4233d7d7453eb60a28900a24155d"
format_stamp: "Formatted at 2026-08-07 00:06:26 on dist-test-slave-pww0"
server_key: "d4a0167283cb668d8b12169f9d863293"
server_key_iv: "556bd6a4bb4560c336f619378b90a15d"
server_key_version: "encryptionkey@0"
I20260807 00:06:26.728045 1129 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:26.730716 1150 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:26.732091 1129 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:26.732227 1129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "b19f4233d7d7453eb60a28900a24155d"
format_stamp: "Formatted at 2026-08-07 00:06:26 on dist-test-slave-pww0"
server_key: "d4a0167283cb668d8b12169f9d863293"
server_key_iv: "556bd6a4bb4560c336f619378b90a15d"
server_key_version: "encryptionkey@0"
I20260807 00:06:26.732343 1129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:26.745942 1129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:26.749049 1129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:26.749274 1129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:26.756628 1129 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:42179
I20260807 00:06:26.756654 1250 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:42179 every 8 connection(s)
I20260807 00:06:26.757714 1129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:26.760936 1251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:26.765537 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1129
I20260807 00:06:26.765676 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:26.765954 31537 external_mini_cluster.cc:1428] Setting key fe8a3c58a9e14ca7a1383cb5b7ac18b9
I20260807 00:06:26.767858 1251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d: Bootstrap starting.
I20260807 00:06:26.770319 1251 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:26.771116 1251 log.cc:824] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:26.773553 1251 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d: No bootstrap required, opened a new log
Aug 07 00:06:26 dist-test-slave-pww0 krb5kdc[1102](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061186, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:26.778299 1251 raft_consensus.cc:348] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19f4233d7d7453eb60a28900a24155d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42179 } }
I20260807 00:06:26.778645 1251 raft_consensus.cc:374] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:26.778776 1251 raft_consensus.cc:729] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b19f4233d7d7453eb60a28900a24155d, State: Initialized, Role: FOLLOWER
I20260807 00:06:26.779341 1251 consensus_queue.cc:260] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19f4233d7d7453eb60a28900a24155d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42179 } }
I20260807 00:06:26.779526 1251 raft_consensus.cc:388] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:26.779601 1251 raft_consensus.cc:482] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:26.779709 1251 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:26.780720 1251 raft_consensus.cc:504] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19f4233d7d7453eb60a28900a24155d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42179 } }
I20260807 00:06:26.781106 1251 leader_election.cc:302] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b19f4233d7d7453eb60a28900a24155d; no voters:
I20260807 00:06:26.781461 1251 leader_election.cc:288] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:26.781569 1257 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:26.781747 1257 raft_consensus.cc:686] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [term 1 LEADER]: Becoming Leader. State: Replica: b19f4233d7d7453eb60a28900a24155d, State: Running, Role: LEADER
I20260807 00:06:26.782079 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:26.767492 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56637 (local address 127.30.204.126:42179)
0807 00:06:26.768043 (+ 551us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:26.768071 (+ 28us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:26.768113 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:06:26.768855 (+ 742us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:26.768875 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:26.769255 (+ 380us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:26.769603 (+ 348us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:26.770507 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:26.771625 (+ 1118us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:26.772836 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:26.773335 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:26.773458 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:26.776449 (+ 2991us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:26.776458 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:26.776483 (+ 25us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:26.776529 (+ 46us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:26.778565 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:26.779199 (+ 634us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:26.779205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:26.779210 (+ 5us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:26.779373 (+ 163us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:26.779755 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:26.779758 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:26.779760 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:26.780230 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:26.780453 (+ 223us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:26.780792 (+ 339us) server_negotiation.cc:299] Negotiation successful
0807 00:06:26.781075 (+ 283us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":370,"thread_start_us":172,"threads_started":1}
I20260807 00:06:26.782145 1257 consensus_queue.cc:237] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19f4233d7d7453eb60a28900a24155d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42179 } }
I20260807 00:06:26.782927 1251 sys_catalog.cc:564] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:26.784178 1257 sys_catalog.cc:455] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [sys.catalog]: SysCatalogTable state changed. Reason: New leader b19f4233d7d7453eb60a28900a24155d. Latest consensus state: current_term: 1 leader_uuid: "b19f4233d7d7453eb60a28900a24155d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19f4233d7d7453eb60a28900a24155d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42179 } } }
I20260807 00:06:26.784319 1257 sys_catalog.cc:458] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:26.784755 1269 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:26.785579 1258 sys_catalog.cc:455] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b19f4233d7d7453eb60a28900a24155d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b19f4233d7d7453eb60a28900a24155d" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 42179 } } }
I20260807 00:06:26.785676 1258 sys_catalog.cc:458] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:26.785862 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:26.787925 1269 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:26.794808 1269 catalog_manager.cc:1347] Generated new cluster ID: 30cfd1a0c7964bdfa9e5d4170a03ef11
I20260807 00:06:26.794943 1269 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:26.796825 1255 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:26.788057 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49933 (local address 127.30.204.126:42179)
0807 00:06:26.788171 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:26.788175 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:26.788541 (+ 366us) server_negotiation.cc:407] Connection header received
0807 00:06:26.788686 (+ 145us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:26.788689 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:26.788748 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:26.788832 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:26.789760 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:26.790363 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:26.791172 (+ 809us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:26.791431 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:26.791552 (+ 121us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:26.793230 (+ 1678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:26.793234 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:26.793236 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:26.793266 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:26.794683 (+ 1417us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:26.795312 (+ 629us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:26.795317 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:26.795319 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:26.795430 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:26.795822 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:26.795826 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:26.795827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:26.796095 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:26.796258 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:26.796527 (+ 269us) server_negotiation.cc:299] Negotiation successful
0807 00:06:26.796642 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":43}
I20260807 00:06:26.813374 1269 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:26.814404 1269 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:26.822955 1269 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b19f4233d7d7453eb60a28900a24155d: Generated new TSK 0
I20260807 00:06:26.823835 1269 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:26.828683 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1129
W20260807 00:06:26.850894 1098 connection.cc:535] client connection to 127.30.204.126:42179 recv error: Network error: failed to read from TLS socket (remote: 127.30.204.126:42179): Connection reset by peer (error 104)
2026-08-07T00:06:26Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1569 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:26 dist-test-slave-pww0 krb5kdc[1295](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:26 dist-test-slave-pww0 krb5kdc[1295](info): set up 2 sockets
Aug 07 00:06:26 dist-test-slave-pww0 krb5kdc[1295](info): commencing operation
krb5kdc: starting...
W20260807 00:06:28.008724 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.143s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:28 dist-test-slave-pww0 krb5kdc[1295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061188, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:28Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:28.146793 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:33337
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:33907
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:33337
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:28.268829 1319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:28.269158 1319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:28.269204 1319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:28.273514 1319 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:28.273600 1319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:28.273619 1319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:28.273636 1319 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:28.273650 1319 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:28.279615 1319 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:33907
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:33337
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:33337
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:28.280826 1319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:28.281955 1319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260807 00:06:28.291491 1319 server_base.cc:1034] running on GCE node
W20260807 00:06:28.291416 1326 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:28.291411 1327 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:28.291410 1329 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:28.292449 1319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:28.293717 1319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:28.294931 1319 hybrid_clock.cc:648] HybridClock initialized: now 1786061188294893 us; error 52 us; skew 500 ppm
Aug 07 00:06:28 dist-test-slave-pww0 krb5kdc[1295](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061188, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:28.298240 1319 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:28.299307 1319 webserver.cc:460] Webserver started at http://127.30.204.126:38165/ using document root <none> and password file <none>
I20260807 00:06:28.299953 1319 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:28.300036 1319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:28.300278 1319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:28.302624 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "56adae7132164bec9976ba17263267c2"
format_stamp: "Formatted at 2026-08-07 00:06:28 on dist-test-slave-pww0"
server_key: "fa63ed89755be6614c677b4cc9ce1802"
server_key_iv: "e79e560f605b1eb9980a0788af1e4aa3"
server_key_version: "encryptionkey@0"
I20260807 00:06:28.303278 1319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "56adae7132164bec9976ba17263267c2"
format_stamp: "Formatted at 2026-08-07 00:06:28 on dist-test-slave-pww0"
server_key: "fa63ed89755be6614c677b4cc9ce1802"
server_key_iv: "e79e560f605b1eb9980a0788af1e4aa3"
server_key_version: "encryptionkey@0"
I20260807 00:06:28.307735 1319 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:06:28.310773 1339 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:28.312436 1319 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:28.312603 1319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "56adae7132164bec9976ba17263267c2"
format_stamp: "Formatted at 2026-08-07 00:06:28 on dist-test-slave-pww0"
server_key: "fa63ed89755be6614c677b4cc9ce1802"
server_key_iv: "e79e560f605b1eb9980a0788af1e4aa3"
server_key_version: "encryptionkey@0"
I20260807 00:06:28.312741 1319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:28.328819 1319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:28.332479 1319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:28.332702 1319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:28.340636 1319 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:33337
I20260807 00:06:28.340667 1414 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:33337 every 8 connection(s)
I20260807 00:06:28.341794 1319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:28.344981 1415 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:28.345329 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1319
I20260807 00:06:28.345472 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:28.345782 31537 external_mini_cluster.cc:1428] Setting key d049c7a35f71cc4b664d5166e3e43228
I20260807 00:06:28.353266 1415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2: Bootstrap starting.
I20260807 00:06:28.356204 1415 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:28.356724 1420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:28.347564 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58867 (local address 127.30.204.126:33337)
0807 00:06:28.348118 (+ 554us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:28.348136 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:28.348178 (+ 42us) server_negotiation.cc:407] Connection header received
0807 00:06:28.349092 (+ 914us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:28.349121 (+ 29us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:28.349546 (+ 425us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:28.349957 (+ 411us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:28.350450 (+ 493us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:28.351649 (+ 1199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:28.352963 (+ 1314us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:28.353561 (+ 598us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:28.353692 (+ 131us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:28.354739 (+ 1047us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:28.354744 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:28.354758 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:28.355199 (+ 441us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:28.355266 (+ 67us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:28.355293 (+ 27us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:28.355405 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:28.355675 (+ 270us) server_negotiation.cc:299] Negotiation successful
0807 00:06:28.355836 (+ 161us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":361,"thread_start_us":152,"threads_started":1}
I20260807 00:06:28.357069 1415 log.cc:824] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:28.359580 1415 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2: No bootstrap required, opened a new log
I20260807 00:06:28.360026 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:28.364045 1415 raft_consensus.cc:348] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56adae7132164bec9976ba17263267c2" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33337 } }
I20260807 00:06:28.364398 1415 raft_consensus.cc:374] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:28.364446 1415 raft_consensus.cc:729] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 56adae7132164bec9976ba17263267c2, State: Initialized, Role: FOLLOWER
I20260807 00:06:28.364916 1415 consensus_queue.cc:260] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56adae7132164bec9976ba17263267c2" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33337 } }
I20260807 00:06:28.365060 1415 raft_consensus.cc:388] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:28.365149 1415 raft_consensus.cc:482] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:28.365255 1415 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:28.366536 1415 raft_consensus.cc:504] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56adae7132164bec9976ba17263267c2" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33337 } }
I20260807 00:06:28.366956 1420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:28.362213 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54405 (local address 127.30.204.126:33337)
0807 00:06:28.362336 (+ 123us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:28.362340 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:28.362615 (+ 275us) server_negotiation.cc:407] Connection header received
0807 00:06:28.362740 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:28.362743 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:28.362807 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:28.362895 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:28.363248 (+ 353us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:28.364058 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:28.365107 (+ 1049us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:28.365473 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:28.365593 (+ 120us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:28.366415 (+ 822us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:28.366419 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:28.366422 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:28.366438 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:28.366473 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:28.366488 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:28.366594 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:28.366771 (+ 177us) server_negotiation.cc:299] Negotiation successful
0807 00:06:28.366813 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:06:28.366976 1415 leader_election.cc:302] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 56adae7132164bec9976ba17263267c2; no voters:
I20260807 00:06:28.367430 1415 leader_election.cc:288] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:28.367558 1430 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:28.367784 1430 raft_consensus.cc:686] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [term 1 LEADER]: Becoming Leader. State: Replica: 56adae7132164bec9976ba17263267c2, State: Running, Role: LEADER
I20260807 00:06:28.368500 1430 consensus_queue.cc:237] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56adae7132164bec9976ba17263267c2" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33337 } }
I20260807 00:06:28.369087 1415 sys_catalog.cc:564] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:28.370419 1432 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "56adae7132164bec9976ba17263267c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56adae7132164bec9976ba17263267c2" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33337 } } }
I20260807 00:06:28.370572 1432 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:28.370922 1431 sys_catalog.cc:455] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 56adae7132164bec9976ba17263267c2. Latest consensus state: current_term: 1 leader_uuid: "56adae7132164bec9976ba17263267c2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "56adae7132164bec9976ba17263267c2" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 33337 } } }
I20260807 00:06:28.371011 1431 sys_catalog.cc:458] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:28.371366 1448 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:28.374490 1448 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:28.380736 1448 catalog_manager.cc:1347] Generated new cluster ID: 09df2c53d94e4c0daad256e434f01e22
I20260807 00:06:28.380837 1448 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:28.392105 1448 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:28.393136 1448 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:28.403090 1448 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 56adae7132164bec9976ba17263267c2: Generated new TSK 0
I20260807 00:06:28.403975 1448 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:28.415940 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1319
W20260807 00:06:28.445914 1291 connection.cc:535] client connection to 127.30.204.126:33337 recv error: Network error: failed to read from TLS socket (remote: 127.30.204.126:33337): Connection reset by peer (error 104)
2026-08-07T00:06:28Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1594 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:28 dist-test-slave-pww0 krb5kdc[1469](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:28 dist-test-slave-pww0 krb5kdc[1469](info): set up 2 sockets
Aug 07 00:06:28 dist-test-slave-pww0 krb5kdc[1469](info): commencing operation
krb5kdc: starting...
W20260807 00:06:29.607514 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.147s user 0.002s sys 0.007s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:29 dist-test-slave-pww0 krb5kdc[1469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061189, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:29.742602 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39911
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:39147
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:39911
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:29.859529 1489 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:29.859822 1489 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:29.859865 1489 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:29.863919 1489 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:29.863991 1489 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:29.864009 1489 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:29.864024 1489 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:29.864038 1489 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:29.870513 1489 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:39147
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:39911
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:39911
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1489
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:29.871709 1489 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:29.872772 1489 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:29.881831 1499 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:29.881835 1496 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:29.881831 1497 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:29.882067 1489 server_base.cc:1034] running on GCE node
I20260807 00:06:29.882704 1489 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:29.884022 1489 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:29.885203 1489 hybrid_clock.cc:648] HybridClock initialized: now 1786061189885190 us; error 41 us; skew 500 ppm
Aug 07 00:06:29 dist-test-slave-pww0 krb5kdc[1469](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061189, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:29.888096 1489 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:29.889087 1489 webserver.cc:460] Webserver started at http://127.30.204.126:34203/ using document root <none> and password file <none>
I20260807 00:06:29.889657 1489 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:29.889725 1489 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:29.889925 1489 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:29.892273 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "77de2c39340a4d648da8025d6f9c7a4c"
format_stamp: "Formatted at 2026-08-07 00:06:29 on dist-test-slave-pww0"
server_key: "c1bcde4f6889a25cf8bba1a60f9a6fc6"
server_key_iv: "3bbb089bbdc74df464217023953c9c99"
server_key_version: "encryptionkey@0"
I20260807 00:06:29.892861 1489 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "77de2c39340a4d648da8025d6f9c7a4c"
format_stamp: "Formatted at 2026-08-07 00:06:29 on dist-test-slave-pww0"
server_key: "c1bcde4f6889a25cf8bba1a60f9a6fc6"
server_key_iv: "3bbb089bbdc74df464217023953c9c99"
server_key_version: "encryptionkey@0"
I20260807 00:06:29.896878 1489 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:29.899493 1507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:29.900847 1489 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:29.900978 1489 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "77de2c39340a4d648da8025d6f9c7a4c"
format_stamp: "Formatted at 2026-08-07 00:06:29 on dist-test-slave-pww0"
server_key: "c1bcde4f6889a25cf8bba1a60f9a6fc6"
server_key_iv: "3bbb089bbdc74df464217023953c9c99"
server_key_version: "encryptionkey@0"
I20260807 00:06:29.901088 1489 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:29.906322 1489 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:29.909355 1489 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:29.909533 1489 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:29.916754 1489 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:39911
I20260807 00:06:29.916785 1598 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:39911 every 8 connection(s)
I20260807 00:06:29.917742 1489 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:29.920635 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1489
I20260807 00:06:29.920722 1601 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:29.920787 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:29.921098 31537 external_mini_cluster.cc:1428] Setting key eb96f46542a38876d2918b8c25b045ec
I20260807 00:06:29.926401 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:29.922507 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53603 (local address 127.30.204.126:39911)
0807 00:06:29.922968 (+ 461us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:29.922986 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:29.923023 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:06:29.923678 (+ 655us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:29.923698 (+ 20us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:29.924003 (+ 305us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:29.924292 (+ 289us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:29.925130 (+ 838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:29.925135 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:29.925147 (+ 12us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:29.925158 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:29.925215 (+ 57us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:29.925239 (+ 24us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:29.925361 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:29.925466 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 00:06:29.925621 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":313,"thread_start_us":147,"threads_started":1}
I20260807 00:06:29.927884 1601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c: Bootstrap starting.
I20260807 00:06:29.929160 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:29.930624 1601 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:29.931627 1601 log.cc:824] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:29.933532 1604 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:29.931658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37997 (local address 127.30.204.126:39911)
0807 00:06:29.931804 (+ 146us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:29.931809 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:29.932021 (+ 212us) server_negotiation.cc:407] Connection header received
0807 00:06:29.932163 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:29.932165 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:29.932199 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:29.932269 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:29.933130 (+ 861us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:29.933133 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:29.933136 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:29.933149 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:29.933173 (+ 24us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:29.933184 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:29.933269 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:29.933367 (+ 98us) server_negotiation.cc:299] Negotiation successful
0807 00:06:29.933409 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260807 00:06:29.934245 1601 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c: No bootstrap required, opened a new log
I20260807 00:06:29.938300 1601 raft_consensus.cc:348] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77de2c39340a4d648da8025d6f9c7a4c" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39911 } }
I20260807 00:06:29.938565 1601 raft_consensus.cc:374] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:29.938620 1601 raft_consensus.cc:729] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 77de2c39340a4d648da8025d6f9c7a4c, State: Initialized, Role: FOLLOWER
I20260807 00:06:29.939041 1601 consensus_queue.cc:260] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77de2c39340a4d648da8025d6f9c7a4c" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39911 } }
I20260807 00:06:29.939164 1601 raft_consensus.cc:388] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:29.939213 1601 raft_consensus.cc:482] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:29.939291 1601 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:29.940523 1601 raft_consensus.cc:504] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77de2c39340a4d648da8025d6f9c7a4c" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39911 } }
I20260807 00:06:29.940910 1601 leader_election.cc:302] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 77de2c39340a4d648da8025d6f9c7a4c; no voters:
I20260807 00:06:29.941231 1601 leader_election.cc:288] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:29.941350 1615 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:29.941581 1615 raft_consensus.cc:686] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [term 1 LEADER]: Becoming Leader. State: Replica: 77de2c39340a4d648da8025d6f9c7a4c, State: Running, Role: LEADER
I20260807 00:06:29.941946 1615 consensus_queue.cc:237] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77de2c39340a4d648da8025d6f9c7a4c" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39911 } }
I20260807 00:06:29.942471 1601 sys_catalog.cc:564] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:29.943614 1617 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 77de2c39340a4d648da8025d6f9c7a4c. Latest consensus state: current_term: 1 leader_uuid: "77de2c39340a4d648da8025d6f9c7a4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77de2c39340a4d648da8025d6f9c7a4c" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39911 } } }
I20260807 00:06:29.943746 1617 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:29.944034 1616 sys_catalog.cc:455] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "77de2c39340a4d648da8025d6f9c7a4c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "77de2c39340a4d648da8025d6f9c7a4c" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 39911 } } }
I20260807 00:06:29.944113 1616 sys_catalog.cc:458] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:29.944175 1630 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:29.947593 1630 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:29.953855 1630 catalog_manager.cc:1347] Generated new cluster ID: 94a986dd9ae24bf692bcf9a7b478f8c9
I20260807 00:06:29.953951 1630 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:29.961987 1630 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:29.962324 1630 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:29.974764 1630 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 77de2c39340a4d648da8025d6f9c7a4c: Generated new TSK 0
I20260807 00:06:29.975666 1630 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:29.988130 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1489
2026-08-07T00:06:29Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1560 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:30 dist-test-slave-pww0 krb5kdc[1649](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:30 dist-test-slave-pww0 krb5kdc[1649](info): set up 2 sockets
Aug 07 00:06:30 dist-test-slave-pww0 krb5kdc[1649](info): commencing operation
krb5kdc: starting...
W20260807 00:06:31.153021 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.132s user 0.000s sys 0.008s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:31 dist-test-slave-pww0 krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:31Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:31.295051 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:38909
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:37965
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:38909
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:31.413765 1666 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:31.414103 1666 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:31.414177 1666 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:31.418452 1666 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:31.418541 1666 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:31.418569 1666 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:31.418591 1666 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:31.418620 1666 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:31.424479 1666 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:37965
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:38909
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:38909
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1666
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:31.425688 1666 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:31.426813 1666 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:31.436365 1673 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:31.436378 1676 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:31.436465 1666 server_base.cc:1034] running on GCE node
W20260807 00:06:31.436365 1674 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:31.437343 1666 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:31.438519 1666 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:31.439683 1666 hybrid_clock.cc:648] HybridClock initialized: now 1786061191439669 us; error 53 us; skew 500 ppm
Aug 07 00:06:31 dist-test-slave-pww0 krb5kdc[1649](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061191, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:31.443336 1666 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:31.444475 1666 webserver.cc:460] Webserver started at http://127.30.204.126:34773/ using document root <none> and password file <none>
I20260807 00:06:31.445107 1666 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:31.445175 1666 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:31.445395 1666 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:31.447764 1666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "8a56c44f9c9242a3ad79204e25522d98"
format_stamp: "Formatted at 2026-08-07 00:06:31 on dist-test-slave-pww0"
server_key: "183f132141e8b18bad21ab2e5bab9e58"
server_key_iv: "3656134f20c6c27bf6bfd3641a54a006"
server_key_version: "encryptionkey@0"
I20260807 00:06:31.448407 1666 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "8a56c44f9c9242a3ad79204e25522d98"
format_stamp: "Formatted at 2026-08-07 00:06:31 on dist-test-slave-pww0"
server_key: "183f132141e8b18bad21ab2e5bab9e58"
server_key_iv: "3656134f20c6c27bf6bfd3641a54a006"
server_key_version: "encryptionkey@0"
I20260807 00:06:31.452677 1666 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.001s
I20260807 00:06:31.455509 1686 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:31.456928 1666 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:31.457077 1666 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "8a56c44f9c9242a3ad79204e25522d98"
format_stamp: "Formatted at 2026-08-07 00:06:31 on dist-test-slave-pww0"
server_key: "183f132141e8b18bad21ab2e5bab9e58"
server_key_iv: "3656134f20c6c27bf6bfd3641a54a006"
server_key_version: "encryptionkey@0"
I20260807 00:06:31.457209 1666 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:31.474545 1666 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:31.478072 1666 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:31.478323 1666 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:31.486279 1666 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:38909
I20260807 00:06:31.486357 1766 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:38909 every 8 connection(s)
I20260807 00:06:31.487373 1666 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:31.490481 1767 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:31.493536 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1666
I20260807 00:06:31.493711 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:31.494024 31537 external_mini_cluster.cc:1428] Setting key 3215390b6bc29ba1870b81047181b472
I20260807 00:06:31.497424 1767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98: Bootstrap starting.
I20260807 00:06:31.500276 1767 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:31.501129 1767 log.cc:824] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:31.503533 1767 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98: No bootstrap required, opened a new log
Aug 07 00:06:31 dist-test-slave-pww0 krb5kdc[1649](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061191, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:31.507616 1767 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a56c44f9c9242a3ad79204e25522d98" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38909 } }
I20260807 00:06:31.507897 1767 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:31.507962 1767 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a56c44f9c9242a3ad79204e25522d98, State: Initialized, Role: FOLLOWER
I20260807 00:06:31.508540 1767 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a56c44f9c9242a3ad79204e25522d98" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38909 } }
I20260807 00:06:31.508780 1767 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:31.508864 1767 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:31.508978 1767 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:31.509684 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:31.495644 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42541 (local address 127.30.204.126:38909)
0807 00:06:31.496156 (+ 512us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:31.496173 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:31.496216 (+ 43us) server_negotiation.cc:407] Connection header received
0807 00:06:31.496902 (+ 686us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:31.496921 (+ 19us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:31.497313 (+ 392us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:31.497697 (+ 384us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:31.498596 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:31.500029 (+ 1433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:31.501023 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:31.501613 (+ 590us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:31.501789 (+ 176us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:31.504545 (+ 2756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:31.504551 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:31.504568 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:31.504604 (+ 36us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:31.506468 (+ 1864us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:31.507004 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:31.507010 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:31.507016 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:31.507162 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:31.507486 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:31.507490 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:31.507492 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:31.507971 (+ 479us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:31.508180 (+ 209us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:31.508475 (+ 295us) server_negotiation.cc:299] Negotiation successful
0807 00:06:31.508795 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":340,"thread_start_us":150,"threads_started":1}
I20260807 00:06:31.510058 1767 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a56c44f9c9242a3ad79204e25522d98" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38909 } }
I20260807 00:06:31.510479 1767 leader_election.cc:302] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8a56c44f9c9242a3ad79204e25522d98; no voters:
I20260807 00:06:31.510855 1767 leader_election.cc:288] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:31.511004 1776 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:31.511260 1776 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [term 1 LEADER]: Becoming Leader. State: Replica: 8a56c44f9c9242a3ad79204e25522d98, State: Running, Role: LEADER
I20260807 00:06:31.511706 1776 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a56c44f9c9242a3ad79204e25522d98" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38909 } }
I20260807 00:06:31.512259 1767 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:31.513434 1778 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8a56c44f9c9242a3ad79204e25522d98. Latest consensus state: current_term: 1 leader_uuid: "8a56c44f9c9242a3ad79204e25522d98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a56c44f9c9242a3ad79204e25522d98" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38909 } } }
I20260807 00:06:31.513576 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:31.513859 1778 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:31.513757 1777 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8a56c44f9c9242a3ad79204e25522d98" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8a56c44f9c9242a3ad79204e25522d98" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 38909 } } }
I20260807 00:06:31.514101 1777 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:31.514376 1792 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:31.518455 1792 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:31.525306 1773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:31.516054 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55729 (local address 127.30.204.126:38909)
0807 00:06:31.516175 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:31.516180 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:31.516362 (+ 182us) server_negotiation.cc:407] Connection header received
0807 00:06:31.516521 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:31.516524 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:31.516588 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:31.516676 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:31.517548 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:31.518328 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:31.519518 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:31.519900 (+ 382us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:31.520019 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:31.521798 (+ 1779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:31.521807 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:31.521810 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:31.521845 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:31.523568 (+ 1723us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:31.524041 (+ 473us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:31.524045 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:31.524047 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:31.524139 (+ 92us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:31.524410 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:31.524413 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:31.524415 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:31.524624 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:31.524744 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:31.525008 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 00:06:31.525137 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260807 00:06:31.525976 1792 catalog_manager.cc:1347] Generated new cluster ID: 4365ad512ddc47c1a88a6048ad70959a
I20260807 00:06:31.526105 1792 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:31.545683 1792 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:31.546733 1792 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:31.557869 1792 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8a56c44f9c9242a3ad79204e25522d98: Generated new TSK 0
I20260807 00:06:31.558681 1792 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:31.564162 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1666
2026-08-07T00:06:31Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1568 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:31 dist-test-slave-pww0 krb5kdc[1815](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:31 dist-test-slave-pww0 krb5kdc[1815](info): set up 2 sockets
Aug 07 00:06:31 dist-test-slave-pww0 krb5kdc[1815](info): commencing operation
krb5kdc: starting...
W20260807 00:06:32.731158 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.142s user 0.003s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:32 dist-test-slave-pww0 krb5kdc[1815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061192, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:32.872499 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:46111
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:41541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:46111
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:32.992074 1836 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:32.992410 1836 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:32.992475 1836 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:32.996729 1836 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:32.996819 1836 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:32.996846 1836 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:32.996868 1836 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:32.996889 1836 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:33.002995 1836 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:41541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:46111
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:46111
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1836
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:33.004376 1836 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:33.005570 1836 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:33.014977 1848 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:33.015061 1843 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:33.015085 1844 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:33.015338 1836 server_base.cc:1034] running on GCE node
I20260807 00:06:33.016011 1836 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:33.017289 1836 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:33.018496 1836 hybrid_clock.cc:648] HybridClock initialized: now 1786061193018450 us; error 61 us; skew 500 ppm
Aug 07 00:06:33 dist-test-slave-pww0 krb5kdc[1815](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061193, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:33.021694 1836 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:33.022734 1836 webserver.cc:460] Webserver started at http://127.30.204.126:35139/ using document root <none> and password file <none>
I20260807 00:06:33.023309 1836 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:33.023376 1836 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:33.023636 1836 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:33.025851 1836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "c47677c975ff4e329c71bbcdc282c16e"
format_stamp: "Formatted at 2026-08-07 00:06:33 on dist-test-slave-pww0"
server_key: "a47339e69c95272f760598c6b8f6df05"
server_key_iv: "12f84dec48fc1ca37675f2b90f6c713e"
server_key_version: "encryptionkey@0"
I20260807 00:06:33.026439 1836 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "c47677c975ff4e329c71bbcdc282c16e"
format_stamp: "Formatted at 2026-08-07 00:06:33 on dist-test-slave-pww0"
server_key: "a47339e69c95272f760598c6b8f6df05"
server_key_iv: "12f84dec48fc1ca37675f2b90f6c713e"
server_key_version: "encryptionkey@0"
I20260807 00:06:33.030547 1836 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.002s sys 0.004s
I20260807 00:06:33.033420 1864 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:33.034837 1836 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:33.034968 1836 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "c47677c975ff4e329c71bbcdc282c16e"
format_stamp: "Formatted at 2026-08-07 00:06:33 on dist-test-slave-pww0"
server_key: "a47339e69c95272f760598c6b8f6df05"
server_key_iv: "12f84dec48fc1ca37675f2b90f6c713e"
server_key_version: "encryptionkey@0"
I20260807 00:06:33.035107 1836 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:33.054298 1836 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:33.058318 1836 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:33.058557 1836 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:33.066895 1836 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:46111
I20260807 00:06:33.066926 1937 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:46111 every 8 connection(s)
I20260807 00:06:33.068091 1836 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:33.070981 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 1836
I20260807 00:06:33.071151 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:33.071533 31537 external_mini_cluster.cc:1428] Setting key 8e5913ccb6bf0d055c2fb2ec92dcf52f
I20260807 00:06:33.071631 1938 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:33.079663 1938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e: Bootstrap starting.
I20260807 00:06:33.081781 1942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:33.073136 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46797 (local address 127.30.204.126:46111)
0807 00:06:33.073638 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:33.073655 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:33.073696 (+ 41us) server_negotiation.cc:407] Connection header received
0807 00:06:33.074680 (+ 984us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:33.074703 (+ 23us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:33.075101 (+ 398us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:33.075432 (+ 331us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:33.075937 (+ 505us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:33.076991 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:33.078268 (+ 1277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:33.078764 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:33.078891 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:33.079923 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:33.079928 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:33.079944 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:33.080373 (+ 429us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:33.080438 (+ 65us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:33.080463 (+ 25us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:33.080541 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:33.080717 (+ 176us) server_negotiation.cc:299] Negotiation successful
0807 00:06:33.080937 (+ 220us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":341,"thread_start_us":153,"threads_started":1}
I20260807 00:06:33.082327 1938 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:33.083228 1938 log.cc:824] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:33.085266 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:33.085899 1938 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e: No bootstrap required, opened a new log
I20260807 00:06:33.090858 1938 raft_consensus.cc:348] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47677c975ff4e329c71bbcdc282c16e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46111 } }
I20260807 00:06:33.091212 1938 raft_consensus.cc:374] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:33.091282 1938 raft_consensus.cc:729] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c47677c975ff4e329c71bbcdc282c16e, State: Initialized, Role: FOLLOWER
I20260807 00:06:33.091831 1938 consensus_queue.cc:260] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47677c975ff4e329c71bbcdc282c16e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46111 } }
I20260807 00:06:33.091992 1938 raft_consensus.cc:388] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:33.092104 1938 raft_consensus.cc:482] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:33.092206 1942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:33.087745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45315 (local address 127.30.204.126:46111)
0807 00:06:33.087883 (+ 138us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:33.087888 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:33.088124 (+ 236us) server_negotiation.cc:407] Connection header received
0807 00:06:33.088227 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:33.088230 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:33.088311 (+ 81us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:33.088413 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:33.088712 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:33.089492 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:33.090414 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:33.090729 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:33.090834 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:33.091715 (+ 881us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:33.091719 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:33.091721 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:33.091736 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:33.091765 (+ 29us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:33.091776 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:33.091840 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:33.091941 (+ 101us) server_negotiation.cc:299] Negotiation successful
0807 00:06:33.092023 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260807 00:06:33.092259 1938 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:33.093355 1938 raft_consensus.cc:504] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47677c975ff4e329c71bbcdc282c16e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46111 } }
I20260807 00:06:33.093762 1938 leader_election.cc:302] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c47677c975ff4e329c71bbcdc282c16e; no voters:
I20260807 00:06:33.094090 1938 leader_election.cc:288] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:33.094228 1957 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:33.094440 1957 raft_consensus.cc:686] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [term 1 LEADER]: Becoming Leader. State: Replica: c47677c975ff4e329c71bbcdc282c16e, State: Running, Role: LEADER
I20260807 00:06:33.094797 1957 consensus_queue.cc:237] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47677c975ff4e329c71bbcdc282c16e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46111 } }
I20260807 00:06:33.095407 1938 sys_catalog.cc:564] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:33.097668 1959 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c47677c975ff4e329c71bbcdc282c16e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47677c975ff4e329c71bbcdc282c16e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46111 } } }
I20260807 00:06:33.097833 1959 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:33.098066 1958 sys_catalog.cc:455] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c47677c975ff4e329c71bbcdc282c16e. Latest consensus state: current_term: 1 leader_uuid: "c47677c975ff4e329c71bbcdc282c16e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c47677c975ff4e329c71bbcdc282c16e" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46111 } } }
I20260807 00:06:33.098536 1958 sys_catalog.cc:458] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e [sys.catalog]: This master's current role is: LEADER
W20260807 00:06:33.101145 1977 catalog_manager.cc:1558] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:33.101231 1977 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:33.101356 1975 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:33.102120 1975 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:33.108433 1975 catalog_manager.cc:1347] Generated new cluster ID: 2ccd9b4d687348d9acd11a658c1e4422
I20260807 00:06:33.108531 1975 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:33.117060 1975 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:33.118491 1975 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:33.123916 1975 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c47677c975ff4e329c71bbcdc282c16e: Generated new TSK 0
I20260807 00:06:33.124847 1975 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:33.132827 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 1836
2026-08-07T00:06:33Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1573 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:33 dist-test-slave-pww0 krb5kdc[1989](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:33 dist-test-slave-pww0 krb5kdc[1989](info): set up 2 sockets
Aug 07 00:06:33 dist-test-slave-pww0 krb5kdc[1989](info): commencing operation
krb5kdc: starting...
W20260807 00:06:34.289294 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.126s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:34 dist-test-slave-pww0 krb5kdc[1989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061194, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:34.426741 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40635
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:32837
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:40635
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:34.547765 2012 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:34.548107 2012 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:34.548184 2012 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:34.552508 2012 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:34.552611 2012 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:34.552634 2012 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:34.552654 2012 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:34.552670 2012 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:34.558384 2012 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:32837
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:40635
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:40635
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2012
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:34.559732 2012 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:34.560848 2012 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:34.570137 2024 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:34.570168 2020 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:34.570241 2012 server_base.cc:1034] running on GCE node
W20260807 00:06:34.570147 2021 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:34.571121 2012 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:34.572393 2012 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:34.573587 2012 hybrid_clock.cc:648] HybridClock initialized: now 1786061194573558 us; error 47 us; skew 500 ppm
Aug 07 00:06:34 dist-test-slave-pww0 krb5kdc[1989](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061194, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:34.576575 2012 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:34.577538 2012 webserver.cc:460] Webserver started at http://127.30.204.126:38337/ using document root <none> and password file <none>
I20260807 00:06:34.578097 2012 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:34.578163 2012 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:34.578344 2012 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:34.580549 2012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4"
format_stamp: "Formatted at 2026-08-07 00:06:34 on dist-test-slave-pww0"
server_key: "52520346ee3dcac7fd6a38cba3f64ea5"
server_key_iv: "b3c7c098616942d006da66e8df1027fa"
server_key_version: "encryptionkey@0"
I20260807 00:06:34.581115 2012 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4"
format_stamp: "Formatted at 2026-08-07 00:06:34 on dist-test-slave-pww0"
server_key: "52520346ee3dcac7fd6a38cba3f64ea5"
server_key_iv: "b3c7c098616942d006da66e8df1027fa"
server_key_version: "encryptionkey@0"
I20260807 00:06:34.585172 2012 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.005s sys 0.000s
I20260807 00:06:34.587792 2033 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:34.589149 2012 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:06:34.589278 2012 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4"
format_stamp: "Formatted at 2026-08-07 00:06:34 on dist-test-slave-pww0"
server_key: "52520346ee3dcac7fd6a38cba3f64ea5"
server_key_iv: "b3c7c098616942d006da66e8df1027fa"
server_key_version: "encryptionkey@0"
I20260807 00:06:34.589386 2012 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:34.594444 2012 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:34.597694 2012 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:34.597858 2012 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:34.604820 2012 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:40635
I20260807 00:06:34.604856 2115 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:40635 every 8 connection(s)
I20260807 00:06:34.605826 2012 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:34.608923 2116 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:34.614705 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2012
I20260807 00:06:34.614833 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:34.615096 31537 external_mini_cluster.cc:1428] Setting key 7878296cc417e0edd74012e189dc648f
I20260807 00:06:34.615159 2116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4: Bootstrap starting.
I20260807 00:06:34.617903 2116 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:34.618777 2116 log.cc:824] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:34.620893 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:34.616686 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39767 (local address 127.30.204.126:40635)
0807 00:06:34.617186 (+ 500us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:34.617201 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:34.617239 (+ 38us) server_negotiation.cc:407] Connection header received
0807 00:06:34.617909 (+ 670us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:34.617927 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:34.618276 (+ 349us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:34.618584 (+ 308us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:34.619495 (+ 911us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:34.619501 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:34.619518 (+ 17us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:34.619532 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:34.619608 (+ 76us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:34.619639 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:34.619756 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:34.619926 (+ 170us) server_negotiation.cc:299] Negotiation successful
0807 00:06:34.620094 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":325,"thread_start_us":133,"threads_started":1}
I20260807 00:06:34.621362 2116 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4: No bootstrap required, opened a new log
I20260807 00:06:34.623832 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:34.625340 2116 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40635 } }
I20260807 00:06:34.625655 2116 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260807 00:06:34.625751 2116 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2c35bd3c6fc94f6bacf793d5940f5ec4, State: Initialized, Role: FOLLOWER
I20260807 00:06:34.626339 2116 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40635 } }
I20260807 00:06:34.626498 2116 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260807 00:06:34.626567 2116 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260807 00:06:34.626678 2116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:34.627672 2119 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:34.625886 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35973 (local address 127.30.204.126:40635)
0807 00:06:34.626012 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:34.626017 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:34.626343 (+ 326us) server_negotiation.cc:407] Connection header received
0807 00:06:34.626466 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:34.626469 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:34.626509 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:34.626578 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:34.627265 (+ 687us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:34.627269 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:34.627272 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0807 00:06:34.627284 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:34.627312 (+ 28us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0807 00:06:34.627323 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:34.627396 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:34.627501 (+ 105us) server_negotiation.cc:299] Negotiation successful
0807 00:06:34.627543 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260807 00:06:34.627777 2116 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40635 } }
I20260807 00:06:34.628177 2116 leader_election.cc:302] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2c35bd3c6fc94f6bacf793d5940f5ec4; no voters:
I20260807 00:06:34.628482 2116 leader_election.cc:288] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260807 00:06:34.628587 2134 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:34.628852 2134 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [term 1 LEADER]: Becoming Leader. State: Replica: 2c35bd3c6fc94f6bacf793d5940f5ec4, State: Running, Role: LEADER
I20260807 00:06:34.629218 2134 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40635 } }
I20260807 00:06:34.629863 2116 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:34.631045 2135 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40635 } } }
I20260807 00:06:34.631189 2135 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:34.631556 2136 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2c35bd3c6fc94f6bacf793d5940f5ec4. Latest consensus state: current_term: 1 leader_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2c35bd3c6fc94f6bacf793d5940f5ec4" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 40635 } } }
I20260807 00:06:34.631664 2136 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:34.631644 2151 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:34.634740 2151 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:34.640727 2151 catalog_manager.cc:1347] Generated new cluster ID: 7cf20ef6ea2c426fa0fa38b4a7756a19
I20260807 00:06:34.640825 2151 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:34.656576 2151 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:34.656987 2151 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:34.662281 2151 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2c35bd3c6fc94f6bacf793d5940f5ec4: Generated new TSK 0
I20260807 00:06:34.663087 2151 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260807 00:06:34.672950 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2012
2026-08-07T00:06:34Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1536 ms)
[----------] 18 tests from AuthTokenIssuingTest (28310 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:34 dist-test-slave-pww0 krb5kdc[2171](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:34 dist-test-slave-pww0 krb5kdc[2171](info): set up 2 sockets
Aug 07 00:06:34 dist-test-slave-pww0 krb5kdc[2171](info): commencing operation
krb5kdc: starting...
W20260807 00:06:35.837177 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.137s user 0.002s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:35 dist-test-slave-pww0 krb5kdc[2171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:35Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:35Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:35.979389 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:46235
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:46235,127.30.204.125:34277,127.30.204.124:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:36.101444 2193 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:06:36.101794 2193 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:36.101835 2193 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:36.101899 2193 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:36.106245 2193 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:36.106326 2193 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:36.106353 2193 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:36.106375 2193 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:36.106406 2193 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:36.112149 2193 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:46235,127.30.204.125:34277,127.30.204.124:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:46235
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2193
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:36.113373 2193 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:36.114472 2193 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:36.123528 2203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:36.123757 2193 server_base.cc:1034] running on GCE node
W20260807 00:06:36.123550 2206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:36.123603 2204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:36.124531 2193 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:36.125788 2193 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:36.126981 2193 hybrid_clock.cc:648] HybridClock initialized: now 1786061196126961 us; error 40 us; skew 500 ppm
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:36.129886 2193 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:36.130896 2193 webserver.cc:460] Webserver started at http://127.30.204.126:33387/ using document root <none> and password file <none>
I20260807 00:06:36.131440 2193 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:36.131548 2193 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:36.131757 2193 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:36.133913 2193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "0412f23d7d144a318bb7b6fb4b31fbba"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "1b4606684f47b95686a11dd1d8fb40d8"
server_key_iv: "e0b74b2169480737cbb76ec12126c1b0"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.134465 2193 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "0412f23d7d144a318bb7b6fb4b31fbba"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "1b4606684f47b95686a11dd1d8fb40d8"
server_key_iv: "e0b74b2169480737cbb76ec12126c1b0"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.138384 2193 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:06:36.141055 2215 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:36.142366 2193 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.001s sys 0.000s
I20260807 00:06:36.142488 2193 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "0412f23d7d144a318bb7b6fb4b31fbba"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "1b4606684f47b95686a11dd1d8fb40d8"
server_key_iv: "e0b74b2169480737cbb76ec12126c1b0"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.142592 2193 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:36.190516 2193 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:36.193691 2193 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:36.193876 2193 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:36.201565 2193 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:46235
I20260807 00:06:36.201840 2313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:46235 every 8 connection(s)
I20260807 00:06:36.202652 2193 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:36.205740 2316 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:36.208783 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2193
I20260807 00:06:36.208935 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:36.209280 31537 external_mini_cluster.cc:1428] Setting key 316c2c42656d937cac8b37fbf2d16af2
I20260807 00:06:36.210660 2316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.204.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.125@KRBTEST.COM" created.
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:36.228338 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.213568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49692 (local address 127.30.204.126:46235)
0807 00:06:36.214100 (+ 532us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.214124 (+ 24us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.214400 (+ 276us) server_negotiation.cc:407] Connection header received
0807 00:06:36.215658 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.215662 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.215869 (+ 207us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.216068 (+ 199us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.217006 (+ 938us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.217920 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.219047 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.219543 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.219733 (+ 190us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.222538 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.222546 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.222562 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.222594 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.224450 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.225284 (+ 834us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.225290 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.225292 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.225432 (+ 140us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.225865 (+ 433us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.225868 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.225869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.226339 (+ 470us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.226464 (+ 125us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.227058 (+ 594us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.227326 (+ 268us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":359,"thread_start_us":153,"threads_started":1}
I20260807 00:06:36.228341 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.213568 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46235 (local address 127.0.0.1:49692)
0807 00:06:36.214237 (+ 669us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.214279 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.215288 (+ 1009us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.216073 (+ 785us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.216079 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.216573 (+ 494us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.216840 (+ 267us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.216856 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.218065 (+ 1209us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.218068 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.218910 (+ 842us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.218923 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.219715 (+ 792us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.219719 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.219859 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.220517 (+ 658us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.220538 (+ 21us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.222335 (+ 1797us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.224630 (+ 2295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.224643 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.224658 (+ 15us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.225026 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.225542 (+ 516us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.225545 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.225547 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.225730 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.226471 (+ 741us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.226480 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.226701 (+ 221us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.226937 (+ 236us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.227431 (+ 494us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":477,"mutex_wait_us":107,"spinlock_wait_cycles":14208,"thread_start_us":177,"threads_started":1}
I20260807 00:06:36.231271 2316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:36.232136 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.231833 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38654)
0807 00:06:36.231977 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.232060 (+ 83us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 00:06:36.232358 2223 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
I20260807 00:06:36.233393 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.233156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38660)
0807 00:06:36.233276 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.233327 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260807 00:06:36.233525 2316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:34277: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab.
Entry for principal kudu/127.30.204.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab.
I20260807 00:06:36.240132 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:34277
--webserver_interface=127.30.204.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab
--principal=kudu/127.30.204.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:46235,127.30.204.125:34277,127.30.204.124:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:06:36.282822 2316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } attempt: 1
I20260807 00:06:36.283878 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.283514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38666)
0807 00:06:36.283698 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.283779 (+ 81us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":65}
I20260807 00:06:36.284822 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.284548 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38678)
0807 00:06:36.284672 (+ 124us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.284737 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":45}
W20260807 00:06:36.284976 2316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:34277: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
I20260807 00:06:36.353170 2316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } attempt: 2
I20260807 00:06:36.354173 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.353787 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38684)
0807 00:06:36.353980 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.354060 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":71}
I20260807 00:06:36.355084 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.354821 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38686)
0807 00:06:36.354941 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.355001 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260807 00:06:36.355330 2316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:34277: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
W20260807 00:06:36.360755 2330 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:06:36.361080 2330 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:36.361104 2330 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:36.361145 2330 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:36.365454 2330 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:36.365537 2330 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:36.365556 2330 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:36.365576 2330 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:36.365589 2330 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:36.371372 2330 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:46235,127.30.204.125:34277,127.30.204.124:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab
--principal=kudu/127.30.204.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:34277
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.204.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2330
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:36.372615 2330 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:36.373698 2330 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:36.382895 2341 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:36.383095 2330 server_base.cc:1034] running on GCE node
W20260807 00:06:36.382912 2343 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:36.382910 2339 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:36.383898 2330 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:36.385141 2330 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:36.386332 2330 hybrid_clock.cc:648] HybridClock initialized: now 1786061196386316 us; error 31 us; skew 500 ppm
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:36.389568 2330 init.cc:377] Logged in from keytab as kudu/127.30.204.125@KRBTEST.COM (short username kudu)
I20260807 00:06:36.390695 2330 webserver.cc:460] Webserver started at http://127.30.204.125:38091/ using document root <none> and password file <none>
I20260807 00:06:36.391284 2330 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:36.391355 2330 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:36.391620 2330 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:36.393900 2330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data/instance:
uuid: "26f0813511fc4a5295f26b836a034c9b"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "113a9efb82f2092873d4315f9ff03216"
server_key_iv: "eeb3570ed0bc07f903c3c3a7adaaf4f0"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.394527 2330 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/wal/instance:
uuid: "26f0813511fc4a5295f26b836a034c9b"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "113a9efb82f2092873d4315f9ff03216"
server_key_iv: "eeb3570ed0bc07f903c3c3a7adaaf4f0"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.398819 2330 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.006s sys 0.000s
I20260807 00:06:36.401679 2350 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:36.403156 2330 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:36.403294 2330 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/wal
uuid: "26f0813511fc4a5295f26b836a034c9b"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "113a9efb82f2092873d4315f9ff03216"
server_key_iv: "eeb3570ed0bc07f903c3c3a7adaaf4f0"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.403420 2330 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:36.446535 2316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } attempt: 3
I20260807 00:06:36.447396 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.447070 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38688)
0807 00:06:36.447225 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.447305 (+ 80us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 00:06:36.448167 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.447949 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38702)
0807 00:06:36.448056 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.448106 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260807 00:06:36.448305 2316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:34277: Network error: Client connection negotiation failed: client connection to 127.30.204.125:34277: connect: Connection refused (error 111)
I20260807 00:06:36.456703 2330 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:36.460207 2330 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:36.460417 2330 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:36.467437 2330 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.125:34277
I20260807 00:06:36.467761 2437 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.125:34277 every 8 connection(s)
I20260807 00:06:36.468524 2330 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
I20260807 00:06:36.468915 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2330
I20260807 00:06:36.469029 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-1/wal/instance
I20260807 00:06:36.469329 31537 external_mini_cluster.cc:1428] Setting key 3b10b4d1a8d8230259fe1b75b5da183c
I20260807 00:06:36.471668 2439 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:36.475915 2439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.204.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.124@KRBTEST.COM" created.
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:36.492383 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.478347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49694 (local address 127.30.204.126:46235)
0807 00:06:36.478521 (+ 174us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.478526 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.479520 (+ 994us) server_negotiation.cc:407] Connection header received
0807 00:06:36.480578 (+ 1058us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.480581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.480639 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.480729 (+ 90us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.482120 (+ 1391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.482895 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.484572 (+ 1677us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.484891 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.485060 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.487865 (+ 2805us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.487870 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.487873 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.487923 (+ 50us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.489665 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.490313 (+ 648us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.490317 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.490318 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.490448 (+ 130us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.490883 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.490887 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.490889 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.491175 (+ 286us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.491297 (+ 122us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.492016 (+ 719us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.492172 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260807 00:06:36.493170 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.478761 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46235 (local address 127.0.0.1:49694)
0807 00:06:36.479299 (+ 538us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.479357 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.480352 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.480921 (+ 569us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.480931 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.481313 (+ 382us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.481928 (+ 615us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.481950 (+ 22us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.483048 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.483052 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.484406 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.484421 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.485081 (+ 660us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.485084 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.485254 (+ 170us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.485899 (+ 645us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.485918 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.487637 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.489833 (+ 2196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.489840 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.489857 (+ 17us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.490175 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.490582 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.490586 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.490587 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.490766 (+ 179us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.491315 (+ 549us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.491321 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.491702 (+ 381us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.491890 (+ 188us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.492201 (+ 311us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":317,"thread_start_us":131,"threads_started":1}
I20260807 00:06:36.494750 2439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab.
Entry for principal kudu/127.30.204.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab.
I20260807 00:06:36.503235 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:37775
--webserver_interface=127.30.204.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab
--principal=kudu/127.30.204.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:38137
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:46235,127.30.204.125:34277,127.30.204.124:37775
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:06:36.504825 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.495285 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38704)
0807 00:06:36.495443 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.495485 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.495585 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.496183 (+ 598us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.496186 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.496338 (+ 152us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.496464 (+ 126us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.496472 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.497626 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.497629 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.498263 (+ 634us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.498273 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.499037 (+ 764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.499041 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.499122 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.499674 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.499688 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.501092 (+ 1404us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.503153 (+ 2061us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.503157 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.503159 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.503416 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.503728 (+ 312us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.503731 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.503733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.503824 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.504286 (+ 462us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.504289 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.504393 (+ 104us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.504487 (+ 94us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.504639 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 00:06:36.504849 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.495420 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38704 (local address 127.30.204.125:34277)
0807 00:06:36.495819 (+ 399us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.495829 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.495850 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:06:36.495887 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.495892 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.496052 (+ 160us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.496166 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.496597 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.497510 (+ 913us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.498398 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.498909 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.499033 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.501251 (+ 2218us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.501264 (+ 13us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.501279 (+ 15us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.501304 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.503029 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.503561 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.503567 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.503568 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.503649 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.503902 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.503904 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.503906 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.504184 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.504297 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.504561 (+ 264us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.504691 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":207,"threads_started":1}
I20260807 00:06:36.506892 2439 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:36.507726 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.507382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53302)
0807 00:06:36.507558 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.507650 (+ 92us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
W20260807 00:06:36.507961 2355 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
I20260807 00:06:36.508992 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.508721 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53312)
0807 00:06:36.508873 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.508929 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":72}
W20260807 00:06:36.509174 2439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37775: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
I20260807 00:06:36.558513 2439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } attempt: 1
I20260807 00:06:36.559549 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.559153 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53326)
0807 00:06:36.559333 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.559403 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":80}
I20260807 00:06:36.560359 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.560116 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53334)
0807 00:06:36.560241 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.560295 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 00:06:36.560518 2439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37775: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
I20260807 00:06:36.591502 2316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } attempt: 4
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:06:36.600777 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.592252 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38708 (local address 127.30.204.125:34277)
0807 00:06:36.592403 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.592408 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.592467 (+ 59us) server_negotiation.cc:407] Connection header received
0807 00:06:36.592646 (+ 179us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.592649 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.592698 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.592765 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.593240 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.594161 (+ 921us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.595332 (+ 1171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.595635 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.595733 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.597789 (+ 2056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.597792 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.597794 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.597821 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.599131 (+ 1310us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.599586 (+ 455us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.599590 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.599593 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.599671 (+ 78us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.599925 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.599928 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.599930 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.600143 (+ 213us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.600227 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.600493 (+ 266us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.600620 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260807 00:06:36.600790 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.592141 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38708)
0807 00:06:36.592376 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.592394 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.592523 (+ 129us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.592803 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.592805 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.593002 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.593118 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.593125 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.594281 (+ 1156us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.594285 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.595204 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.595215 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.595755 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.595758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.595860 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.596350 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.596364 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.597642 (+ 1278us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.599264 (+ 1622us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.599267 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.599270 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.599498 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.599754 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.599757 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.599759 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.599843 (+ 84us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.600234 (+ 391us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.600237 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.600340 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.600428 (+ 88us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.600586 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131}
I20260807 00:06:36.601701 2316 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:36.602465 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.602172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53350)
0807 00:06:36.602323 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.602383 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260807 00:06:36.603559 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.603192 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53354)
0807 00:06:36.603395 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.603486 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":118}
W20260807 00:06:36.603677 2316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37775: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
W20260807 00:06:36.624570 2450 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:06:36.624933 2450 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:36.624967 2450 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:36.625041 2450 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 00:06:36.628708 2439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } attempt: 2
I20260807 00:06:36.629719 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.629336 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53366)
0807 00:06:36.629535 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.629630 (+ 95us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
W20260807 00:06:36.629709 2450 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:36.629778 2450 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:36.629805 2450 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:36.629829 2450 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:36.629858 2450 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:36.630614 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.630340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53374)
0807 00:06:36.630478 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.630528 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260807 00:06:36.630839 2439 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37775: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
I20260807 00:06:36.635823 2450 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:38137
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:46235,127.30.204.125:34277,127.30.204.124:37775
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab
--principal=kudu/127.30.204.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:37775
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.204.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2450
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:36.637012 2450 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:36.638161 2450 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:36.647419 2463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:36.647419 2462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:36.647576 2450 server_base.cc:1034] running on GCE node
W20260807 00:06:36.647418 2466 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:36.648471 2450 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:36.649657 2450 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:36.650842 2450 hybrid_clock.cc:648] HybridClock initialized: now 1786061196650823 us; error 37 us; skew 500 ppm
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:36.654179 2450 init.cc:377] Logged in from keytab as kudu/127.30.204.124@KRBTEST.COM (short username kudu)
I20260807 00:06:36.655311 2450 webserver.cc:460] Webserver started at http://127.30.204.124:45059/ using document root <none> and password file <none>
I20260807 00:06:36.655993 2450 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:36.656080 2450 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:36.656316 2450 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:36.658643 2450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data/instance:
uuid: "4e99c44d4e854f509eb06e8d930b66a2"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "ba9bc3315afe34a97423d83acb62af3a"
server_key_iv: "a2437f61fa8fca08bf46962f155f8aa3"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.659274 2450 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/wal/instance:
uuid: "4e99c44d4e854f509eb06e8d930b66a2"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "ba9bc3315afe34a97423d83acb62af3a"
server_key_iv: "a2437f61fa8fca08bf46962f155f8aa3"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.662846 2316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } attempt: 1
I20260807 00:06:36.663618 2450 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:36.663688 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.663320 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53386)
0807 00:06:36.663512 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.663591 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 00:06:36.664535 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.664283 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53392)
0807 00:06:36.664394 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.664455 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 00:06:36.664745 2316 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37775: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37775: connect: Connection refused (error 111)
I20260807 00:06:36.666589 2475 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:36.668015 2450 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:36.668211 2450 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/wal
uuid: "4e99c44d4e854f509eb06e8d930b66a2"
format_stamp: "Formatted at 2026-08-07 00:06:36 on dist-test-slave-pww0"
server_key: "ba9bc3315afe34a97423d83acb62af3a"
server_key_iv: "a2437f61fa8fca08bf46962f155f8aa3"
server_key_version: "encryptionkey@0"
I20260807 00:06:36.668349 2450 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:36.695545 2450 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:36.699714 2450 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:36.699887 2450 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:36.707060 2450 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.124:37775
I20260807 00:06:36.707377 2549 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.124:37775 every 8 connection(s)
I20260807 00:06:36.708222 2450 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
I20260807 00:06:36.711217 2550 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:36.712185 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2450
I20260807 00:06:36.712296 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1786060901439409-31537-0/minicluster-data/master-2/wal/instance
I20260807 00:06:36.712571 31537 external_mini_cluster.cc:1428] Setting key 90b1e91b70d41e835e09f210e1488510
I20260807 00:06:36.716123 2550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:36.722012 2439 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } attempt: 3
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
I20260807 00:06:36.728659 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.713985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53404 (local address 127.30.204.124:37775)
0807 00:06:36.714490 (+ 505us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.714511 (+ 21us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.714559 (+ 48us) server_negotiation.cc:407] Connection header received
0807 00:06:36.715410 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.715436 (+ 26us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.715922 (+ 486us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.716330 (+ 408us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.717119 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.718487 (+ 1368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.719591 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.720247 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.720402 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.723291 (+ 2889us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.723298 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.723314 (+ 16us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.723340 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.725195 (+ 1855us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.725746 (+ 551us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.725753 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.725761 (+ 8us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.725919 (+ 158us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.726265 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.726268 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.726270 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.726871 (+ 601us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.727101 (+ 230us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.727397 (+ 296us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.727722 (+ 325us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":336,"thread_start_us":140,"threads_started":1}
I20260807 00:06:36.729003 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.718304 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49710 (local address 127.30.204.126:46235)
0807 00:06:36.718454 (+ 150us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.718458 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.718891 (+ 433us) server_negotiation.cc:407] Connection header received
0807 00:06:36.719021 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.719025 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.719081 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.719164 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.719928 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.720602 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.721823 (+ 1221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.722072 (+ 249us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.722174 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.725066 (+ 2892us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.725070 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.725073 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.725108 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.726787 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.727353 (+ 566us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.727357 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.727359 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.727492 (+ 133us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.727975 (+ 483us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.727979 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.727981 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.728233 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.728308 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.728735 (+ 427us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.728851 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:06:36.729436 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.718249 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:46235 (local address 127.0.0.1:49710)
0807 00:06:36.718732 (+ 483us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.718790 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.718927 (+ 137us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.719225 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.719230 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.719664 (+ 434us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.719805 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.719812 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.720732 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.720734 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.721689 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.721704 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.722255 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.722259 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.722372 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.722891 (+ 519us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.722915 (+ 24us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.724892 (+ 1977us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.726920 (+ 2028us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.726930 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.726937 (+ 7us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.727225 (+ 288us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.727768 (+ 543us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.727771 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.727773 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.727867 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.728392 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.728399 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.728526 (+ 127us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.728658 (+ 132us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.728847 (+ 189us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":277,"thread_start_us":165,"threads_started":1}
I20260807 00:06:36.730625 2550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:36.731891 2316 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } attempt: 2
I20260807 00:06:36.733664 2444 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.722490 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53410)
0807 00:06:36.722903 (+ 413us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.722916 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.722983 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.723577 (+ 594us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.723580 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.723772 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.723890 (+ 118us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.723898 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.725092 (+ 1194us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.725096 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.725761 (+ 665us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.725772 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.726330 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.726335 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.726435 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.726892 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.726909 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.728391 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.731024 (+ 2633us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.731028 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.731030 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.731279 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.731623 (+ 344us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.731627 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.731629 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.731726 (+ 97us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.733164 (+ 1438us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.733167 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.733283 (+ 116us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.733365 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.733498 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":326}
I20260807 00:06:36.734311 2558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.722547 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53410 (local address 127.30.204.124:37775)
0807 00:06:36.723302 (+ 755us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.723314 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.723326 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:06:36.723352 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.723354 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.723394 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.723507 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.724023 (+ 516us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.724994 (+ 971us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.725872 (+ 878us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.726213 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.726317 (+ 104us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.728526 (+ 2209us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.728529 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.728532 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.728558 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.730876 (+ 2318us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.731361 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.731364 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.731366 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.731508 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.731846 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.731849 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.731850 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.733048 (+ 1198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.733935 (+ 887us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.734017 (+ 82us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.734131 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":690,"thread_start_us":571,"threads_started":1}
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:06:36.738015 2439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: Bootstrap starting.
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:36.741274 2439 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:36.742007 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.731199 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38722 (local address 127.30.204.125:34277)
0807 00:06:36.731341 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.731346 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.731601 (+ 255us) server_negotiation.cc:407] Connection header received
0807 00:06:36.731710 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.731713 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.731757 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.731819 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.732225 (+ 406us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.732935 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.734228 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.734548 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.734620 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.738031 (+ 3411us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.738035 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.738038 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.738063 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.739611 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.740527 (+ 916us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.740531 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.740533 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.740638 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.740966 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.740969 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.740971 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.741258 (+ 287us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.741327 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.741716 (+ 389us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.741844 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":57}
I20260807 00:06:36.742338 2439 log.cc:824] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:36.742539 2324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.732217 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53416)
0807 00:06:36.732477 (+ 260us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.732490 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.732557 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.732825 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.732828 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.733014 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.733109 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.733117 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.734017 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.734020 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.734742 (+ 722us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.734752 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.735370 (+ 618us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.735373 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.735488 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.735876 (+ 388us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.735890 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.738444 (+ 2554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.740948 (+ 2504us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.740951 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.740953 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.741207 (+ 254us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.741531 (+ 324us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.741534 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.741536 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.741630 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.742059 (+ 429us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.742063 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.742161 (+ 98us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.742242 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.742383 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":183,"spinlock_wait_cycles":1920}
I20260807 00:06:36.742662 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.731102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:34277 (local address 127.0.0.1:38722)
0807 00:06:36.731532 (+ 430us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.731545 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.731638 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.731843 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.731845 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.732004 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.732123 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.732129 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.733068 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.733072 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.734101 (+ 1029us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.734121 (+ 20us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.735049 (+ 928us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.735052 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.735216 (+ 164us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.735905 (+ 689us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.735919 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.737892 (+ 1973us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.739733 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.739736 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.739739 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.740398 (+ 659us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.740760 (+ 362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.740762 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.740765 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.740855 (+ 90us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.741401 (+ 546us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.741404 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.741510 (+ 106us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.742326 (+ 816us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.742495 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":335,"mutex_wait_us":32}
I20260807 00:06:36.743140 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.732450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53416 (local address 127.30.204.124:37775)
0807 00:06:36.732660 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.732663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.732671 (+ 8us) server_negotiation.cc:407] Connection header received
0807 00:06:36.732695 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.732697 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.732728 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.732786 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.733222 (+ 436us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.733900 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.734881 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.735243 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.735692 (+ 449us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.738600 (+ 2908us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.738603 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.738606 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.738631 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.740823 (+ 2192us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.741336 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.741339 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.741341 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.741420 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.741749 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.741751 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.741753 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.741951 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.742837 (+ 886us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.742912 (+ 75us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.743010 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":25,"server-negotiator.queue_time_us":154}
I20260807 00:06:36.743521 2550 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:36.743948 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.731967 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49714 (local address 127.30.204.126:46235)
0807 00:06:36.732075 (+ 108us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.732078 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.732247 (+ 169us) server_negotiation.cc:407] Connection header received
0807 00:06:36.732358 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.732360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.732409 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.732477 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.733303 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.734038 (+ 735us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.735129 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.735413 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.735570 (+ 157us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.739157 (+ 3587us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.739161 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.739164 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.739190 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.740663 (+ 1473us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.741149 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.741152 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.741154 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.741243 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.741653 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.741656 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.741658 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.741883 (+ 225us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.742531 (+ 648us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.743684 (+ 1153us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.743809 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260807 00:06:36.746106 2439 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: No bootstrap required, opened a new log
I20260807 00:06:36.746452 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba: Bootstrap starting.
I20260807 00:06:36.749524 2316 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:36.750372 2439 raft_consensus.cc:348] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
I20260807 00:06:36.750568 2316 log.cc:824] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:36.750666 2439 raft_consensus.cc:729] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 26f0813511fc4a5295f26b836a034c9b, State: Initialized, Role: FOLLOWER
I20260807 00:06:36.751055 2439 consensus_queue.cc:260] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061195, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:06:36.752890 2439 sys_catalog.cc:564] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:36.753526 2316 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba: No bootstrap required, opened a new log
Aug 07 00:06:36 dist-test-slave-pww0 krb5kdc[2171](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061196, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
I20260807 00:06:36.758062 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.745195 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38736 (local address 127.30.204.125:34277)
0807 00:06:36.745314 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.745319 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.745488 (+ 169us) server_negotiation.cc:407] Connection header received
0807 00:06:36.745765 (+ 277us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.745769 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.745818 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.745868 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.746824 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.747579 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.748721 (+ 1142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.749025 (+ 304us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.749099 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.752688 (+ 3589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.752692 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.752695 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.752725 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.754363 (+ 1638us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.754925 (+ 562us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.754929 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.754931 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.755020 (+ 89us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.756947 (+ 1927us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.756951 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.756953 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.757194 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.757349 (+ 155us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.757773 (+ 424us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.757905 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:06:36.758031 2316 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
I20260807 00:06:36.758297 2316 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0412f23d7d144a318bb7b6fb4b31fbba, State: Initialized, Role: FOLLOWER
I20260807 00:06:36.758407 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.744555 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53432 (local address 127.30.204.124:37775)
0807 00:06:36.744756 (+ 201us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.744761 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.747664 (+ 2903us) server_negotiation.cc:407] Connection header received
0807 00:06:36.747974 (+ 310us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.747978 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.748034 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.748092 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.748731 (+ 639us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.749463 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.750541 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.750832 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.750915 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.754819 (+ 3904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.754824 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.754828 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.754856 (+ 28us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.756535 (+ 1679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.757052 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.757056 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.757057 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.757150 (+ 93us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.757470 (+ 320us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.757473 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.757474 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.757720 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.757800 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.758110 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.758235 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115}
I20260807 00:06:36.758668 2316 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
I20260807 00:06:36.758855 2556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.744059 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37775 (local address 127.0.0.1:53432)
0807 00:06:36.747591 (+ 3532us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:36.747608 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:36.747807 (+ 199us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:36.748242 (+ 435us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:36.748245 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:36.748454 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:36.748595 (+ 141us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.748604 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.749620 (+ 1016us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.749623 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.750386 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:36.750399 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.751073 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.751077 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:36.751156 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.751678 (+ 522us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:36.751695 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:36.754646 (+ 2951us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:36.756651 (+ 2005us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.756655 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.756657 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.756928 (+ 271us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.757263 (+ 335us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:36.757266 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:36.757268 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:36.757363 (+ 95us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:36.757864 (+ 501us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:36.757867 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:36.757950 (+ 83us) client_negotiation.cc:769] Sending connection context
0807 00:06:36.758554 (+ 604us) client_negotiation.cc:240] Negotiation successful
0807 00:06:36.758697 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3442}
I20260807 00:06:36.761566 2316 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:36.761962 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
W20260807 00:06:36.762326 2581 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:36.762428 2581 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:36.762444 2550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: Bootstrap starting.
I20260807 00:06:36.763367 2267 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "0412f23d7d144a318bb7b6fb4b31fbba"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:49714
I20260807 00:06:36.763672 2267 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 00:06:36.763803 2267 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:36.764863 2564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.765137 2564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:36.765350 2267 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
I20260807 00:06:36.765460 2550 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:36.757484 2330 master_runner.cc:186] Error getting master registration for 127.30.204.126:46235: OK, instance_id { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" instance_seqno: 1786061196199424 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:06:36.766000 2330 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:06:36.766537 2550 log.cc:824] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:36.766979 2395 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "26f0813511fc4a5295f26b836a034c9b"
I20260807 00:06:36.767256 2395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:36.768836 2395 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0412f23d7d144a318bb7b6fb4b31fbba in term 1.
I20260807 00:06:36.769397 2223 leader_election.cc:302] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 0412f23d7d144a318bb7b6fb4b31fbba, 26f0813511fc4a5295f26b836a034c9b; no voters:
I20260807 00:06:36.766849 2267 leader_election.cc:288] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [CANDIDATE]: Term 1 election: Requested vote from peers 26f0813511fc4a5295f26b836a034c9b (127.30.204.125:34277), 4e99c44d4e854f509eb06e8d930b66a2 (127.30.204.124:37775)
I20260807 00:06:36.767038 2584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.769760 2584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:36.769878 2584 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:36.769968 2584 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [term 1 LEADER]: Becoming Leader. State: Replica: 0412f23d7d144a318bb7b6fb4b31fbba, State: Running, Role: LEADER
I20260807 00:06:36.770376 2584 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
W20260807 00:06:36.770634 2600 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:36.770714 2600 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:36.776548 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0412f23d7d144a318bb7b6fb4b31fbba. Latest consensus state: current_term: 1 leader_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.776741 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:36.781466 2193 master_runner.cc:186] Error getting master registration for 127.30.204.124:37775: OK, instance_id { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" instance_seqno: 1786061196704916 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:06:36.785892 2193 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:06:36.788728 2611 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:36.789844 2611 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260807 00:06:36.797376 2550 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: No bootstrap required, opened a new log
I20260807 00:06:36.798163 2447 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.775891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38750 (local address 127.30.204.125:34277)
0807 00:06:36.776025 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.776030 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.787891 (+ 11861us) server_negotiation.cc:407] Connection header received
0807 00:06:36.788028 (+ 137us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.788032 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.788093 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.788178 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.789351 (+ 1173us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.790076 (+ 725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.791320 (+ 1244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.791657 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.791772 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.793652 (+ 1880us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.793656 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.793658 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.793693 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.796046 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.796570 (+ 524us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.796574 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.796576 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.796687 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.796998 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.797002 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.797004 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.797345 (+ 341us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.797466 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.797849 (+ 383us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.797987 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260807 00:06:36.809757 2554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.779369 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53436 (local address 127.30.204.124:37775)
0807 00:06:36.779513 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.779517 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.798839 (+ 19322us) server_negotiation.cc:407] Connection header received
0807 00:06:36.800302 (+ 1463us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.800306 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.800359 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.800408 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.801307 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.802106 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.803287 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.803689 (+ 402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.803773 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.805593 (+ 1820us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.805597 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.805600 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.805632 (+ 32us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.807221 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.808230 (+ 1009us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.808234 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.808236 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.808330 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.808706 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.808710 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.808712 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.808945 (+ 233us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.809063 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.809430 (+ 367us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.809580 (+ 150us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:06:36.812865 2323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:36.776387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49718 (local address 127.30.204.126:46235)
0807 00:06:36.776702 (+ 315us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.776706 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.788237 (+ 11531us) server_negotiation.cc:407] Connection header received
0807 00:06:36.788413 (+ 176us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.788416 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.788472 (+ 56us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.788543 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.792455 (+ 3912us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.793239 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.794278 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:36.794563 (+ 285us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:36.794682 (+ 119us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:36.806586 (+ 11904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:36.806590 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:36.806593 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:36.806628 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:36.808408 (+ 1780us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.810223 (+ 1815us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.810228 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.810230 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.810362 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:36.811178 (+ 816us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:36.811182 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:36.811184 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:36.811437 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:36.811599 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:36.812549 (+ 950us) server_negotiation.cc:299] Negotiation successful
0807 00:06:36.812689 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":227}
I20260807 00:06:36.813663 2395 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [term 1 FOLLOWER]: Refusing update from remote peer 0412f23d7d144a318bb7b6fb4b31fbba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:36.814549 2603 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [LEADER]: Connected to new peer: Peer: permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:36.816406 2550 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
I20260807 00:06:36.819554 2550 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4e99c44d4e854f509eb06e8d930b66a2, State: Initialized, Role: FOLLOWER
I20260807 00:06:36.819676 2564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0412f23d7d144a318bb7b6fb4b31fbba. Latest consensus state: current_term: 1 leader_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.819828 2564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:36.820369 2550 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } }
I20260807 00:06:36.822840 2517 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "4e99c44d4e854f509eb06e8d930b66a2"
I20260807 00:06:36.823123 2516 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:36.823298 2550 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:36.824739 2516 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [term 1 FOLLOWER]: Refusing update from remote peer 0412f23d7d144a318bb7b6fb4b31fbba: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:36.826675 2517 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 0412f23d7d144a318bb7b6fb4b31fbba in term 1.
I20260807 00:06:36.827302 2624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.828761 2624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:36.828951 2603 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [LEADER]: Connected to new peer: Peer: permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:36.836359 2624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0412f23d7d144a318bb7b6fb4b31fbba. Latest consensus state: current_term: 1 leader_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.836522 2624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:36.838076 2584 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.838243 2584 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: This master's current role is: LEADER
W20260807 00:06:36.838636 2647 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:36.838773 2647 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:36.839322 2564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.839485 2564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:36.842032 2619 mvcc.cc:205] Tried to move back new op lower bound from 7315706662120697856 to 7315706661975474176. Current Snapshot: MvccSnapshot[applied={T|T < 7315706662120697856}]
I20260807 00:06:36.846530 2611 catalog_manager.cc:1347] Generated new cluster ID: bfe0bdd05ec1427c86222e657b816b5c
I20260807 00:06:36.846652 2611 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:36.854063 2603 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.854238 2603 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:36.856364 2624 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "0412f23d7d144a318bb7b6fb4b31fbba" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 46235 } } peers { permanent_uuid: "26f0813511fc4a5295f26b836a034c9b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 34277 } } peers { permanent_uuid: "4e99c44d4e854f509eb06e8d930b66a2" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37775 } } }
I20260807 00:06:36.856523 2624 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:36.880026 2611 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:36.881466 2611 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:36.906625 2611 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0412f23d7d144a318bb7b6fb4b31fbba: Generated new TSK 0
I20260807 00:06:36.907702 2611 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:06:36.945995 2323 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.942121 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49726 (local address 127.30.204.126:46235)
0807 00:06:36.942266 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.942271 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.944748 (+ 2477us) server_negotiation.cc:407] Connection header received
0807 00:06:36.944987 (+ 239us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.944992 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.945049 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.945104 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.945848 (+ 744us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49726: BlockingRecv error: recv got EOF from 127.0.0.1:49726 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260807 00:06:36.948815 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.943127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38764 (local address 127.30.204.125:34277)
0807 00:06:36.943256 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.943260 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.946955 (+ 3695us) server_negotiation.cc:407] Connection header received
0807 00:06:36.947721 (+ 766us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.947725 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.947788 (+ 63us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.947845 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.948655 (+ 810us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38764: BlockingRecv error: recv got EOF from 127.0.0.1:38764 (error 108)
Metrics: {"server-negotiator.queue_time_us":51}
W20260807 00:06:36.949975 2554 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.944002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53450 (local address 127.30.204.124:37775)
0807 00:06:36.944135 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.944139 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.947182 (+ 3043us) server_negotiation.cc:407] Connection header received
0807 00:06:36.949106 (+ 1924us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.949110 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.949171 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.949226 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.949838 (+ 612us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53450: BlockingRecv error: recv got EOF from 127.0.0.1:53450 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 00:06:36.955832 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.953828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38772 (local address 127.30.204.125:34277)
0807 00:06:36.953954 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.953959 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.954536 (+ 577us) server_negotiation.cc:407] Connection header received
0807 00:06:36.954822 (+ 286us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.954826 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.954915 (+ 89us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.954979 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.955705 (+ 726us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38772: BlockingRecv error: recv got EOF from 127.0.0.1:38772 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 00:06:36.961760 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.959835 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38786 (local address 127.30.204.125:34277)
0807 00:06:36.959960 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.959964 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.960624 (+ 660us) server_negotiation.cc:407] Connection header received
0807 00:06:36.960875 (+ 251us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.960879 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.960938 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.960995 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.961616 (+ 621us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38786: BlockingRecv error: recv got EOF from 127.0.0.1:38786 (error 108)
Metrics: {"server-negotiator.queue_time_us":49}
W20260807 00:06:36.969302 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.967245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38790 (local address 127.30.204.125:34277)
0807 00:06:36.967393 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.967398 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.968143 (+ 745us) server_negotiation.cc:407] Connection header received
0807 00:06:36.968407 (+ 264us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.968412 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.968476 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.968537 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.969172 (+ 635us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38790: BlockingRecv error: recv got EOF from 127.0.0.1:38790 (error 108)
Metrics: {"server-negotiator.queue_time_us":66}
W20260807 00:06:36.977638 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.975807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38796 (local address 127.30.204.125:34277)
0807 00:06:36.975962 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.975967 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.976598 (+ 631us) server_negotiation.cc:407] Connection header received
0807 00:06:36.976817 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.976821 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.976876 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.976926 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.977512 (+ 586us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38796: BlockingRecv error: recv got EOF from 127.0.0.1:38796 (error 108)
Metrics: {"server-negotiator.queue_time_us":79}
W20260807 00:06:36.990389 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:36.988423 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38800 (local address 127.30.204.125:34277)
0807 00:06:36.988555 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:36.988559 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:36.989231 (+ 672us) server_negotiation.cc:407] Connection header received
0807 00:06:36.989443 (+ 212us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:36.989446 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:36.989501 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:36.989551 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:36.990233 (+ 682us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38800: BlockingRecv error: recv got EOF from 127.0.0.1:38800 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 00:06:37.010421 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:37.009063 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38810 (local address 127.30.204.125:34277)
0807 00:06:37.009239 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:37.009245 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:37.009406 (+ 161us) server_negotiation.cc:407] Connection header received
0807 00:06:37.009588 (+ 182us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:37.009592 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:37.009669 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:37.009799 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:37.010256 (+ 457us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38810: BlockingRecv error: recv got EOF from 127.0.0.1:38810 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 00:06:37.046248 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:37.044880 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38816 (local address 127.30.204.125:34277)
0807 00:06:37.045112 (+ 232us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:37.045118 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:37.045236 (+ 118us) server_negotiation.cc:407] Connection header received
0807 00:06:37.045390 (+ 154us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:37.045393 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:37.045453 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:37.045539 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:37.046130 (+ 591us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38816: BlockingRecv error: recv got EOF from 127.0.0.1:38816 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
W20260807 00:06:37.113821 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:37.112654 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38828 (local address 127.30.204.125:34277)
0807 00:06:37.112836 (+ 182us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:37.112840 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:37.112970 (+ 130us) server_negotiation.cc:407] Connection header received
0807 00:06:37.113079 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:37.113082 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:37.113137 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:37.113214 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:37.113693 (+ 479us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38828: BlockingRecv error: recv got EOF from 127.0.0.1:38828 (error 108)
Metrics: {"server-negotiator.queue_time_us":76}
W20260807 00:06:37.245175 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:37.244046 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38830 (local address 127.30.204.125:34277)
0807 00:06:37.244226 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:37.244231 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:37.244307 (+ 76us) server_negotiation.cc:407] Connection header received
0807 00:06:37.244450 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:37.244454 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:37.244519 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:37.244636 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:37.245067 (+ 431us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38830: BlockingRecv error: recv got EOF from 127.0.0.1:38830 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
W20260807 00:06:37.504806 2447 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:37.503646 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38844 (local address 127.30.204.125:34277)
0807 00:06:37.503862 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:37.503867 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:37.503947 (+ 80us) server_negotiation.cc:407] Connection header received
0807 00:06:37.504089 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:37.504091 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:37.504141 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:37.504222 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:37.504654 (+ 432us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:38844: BlockingRecv error: recv got EOF from 127.0.0.1:38844 (error 108)
Metrics: {"server-negotiator.queue_time_us":120}
I20260807 00:06:37.763970 2581 catalog_manager.cc:1259] Loaded cluster ID: bfe0bdd05ec1427c86222e657b816b5c
I20260807 00:06:37.764058 2581 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: loading cluster ID for follower catalog manager: success
I20260807 00:06:37.765823 2581 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: acquiring CA information for follower catalog manager: success
I20260807 00:06:37.767110 2581 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 26f0813511fc4a5295f26b836a034c9b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:06:37.840390 2647 catalog_manager.cc:1259] Loaded cluster ID: bfe0bdd05ec1427c86222e657b816b5c
I20260807 00:06:37.840494 2647 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: loading cluster ID for follower catalog manager: success
I20260807 00:06:37.842324 2647 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: acquiring CA information for follower catalog manager: success
I20260807 00:06:37.843734 2647 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 4e99c44d4e854f509eb06e8d930b66a2: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:06:38.023649 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:38.019191 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38852 (local address 127.30.204.125:34277)
0807 00:06:38.019450 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:38.019483 (+ 33us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:38.019504 (+ 21us) server_negotiation.cc:407] Connection header received
0807 00:06:38.019723 (+ 219us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:38.019727 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:38.019875 (+ 148us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:38.019982 (+ 107us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:38.020326 (+ 344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.020878 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.021828 (+ 950us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.022088 (+ 260us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.022183 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:38.022620 (+ 437us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:38.023098 (+ 478us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:38.023204 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:38.023365 (+ 161us) server_negotiation.cc:299] Negotiation successful
0807 00:06:38.023436 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":191,"thread_start_us":84,"threads_started":1}
I20260807 00:06:38.031293 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:38.027258 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53454 (local address 127.30.204.124:37775)
0807 00:06:38.027557 (+ 299us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:38.027562 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:38.027635 (+ 73us) server_negotiation.cc:407] Connection header received
0807 00:06:38.027855 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:38.027858 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:38.027987 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:38.028097 (+ 110us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:38.028342 (+ 245us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.028865 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.029731 (+ 866us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.029964 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.030057 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:38.030406 (+ 349us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:38.030862 (+ 456us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:38.030979 (+ 117us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:38.031047 (+ 68us) server_negotiation.cc:299] Negotiation successful
0807 00:06:38.031115 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":218,"thread_start_us":80,"threads_started":1}
I20260807 00:06:38.039021 2780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:38.035220 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53466 (local address 127.30.204.124:37775)
0807 00:06:38.035338 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:38.035342 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:38.035596 (+ 254us) server_negotiation.cc:407] Connection header received
0807 00:06:38.035809 (+ 213us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:38.035812 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:38.035858 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:38.035920 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:38.036185 (+ 265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.036795 (+ 610us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.037878 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.038218 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.038307 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:38.038538 (+ 231us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:38.038636 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:38.038706 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:38.038808 (+ 102us) server_negotiation.cc:299] Negotiation successful
0807 00:06:38.038869 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260807 00:06:38.039119 2767 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:38.034910 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38868 (local address 127.30.204.125:34277)
0807 00:06:38.035043 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:38.035047 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:38.035105 (+ 58us) server_negotiation.cc:407] Connection header received
0807 00:06:38.035313 (+ 208us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:38.035317 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:38.035382 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:38.035488 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:38.035746 (+ 258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.036526 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.037823 (+ 1297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.038078 (+ 255us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.038180 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:38.038456 (+ 276us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:38.038554 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:38.038631 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:38.038941 (+ 310us) server_negotiation.cc:299] Negotiation successful
0807 00:06:38.038998 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260807 00:06:38.040100 2789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:38.034847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49728 (local address 127.30.204.126:46235)
0807 00:06:38.035114 (+ 267us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:38.035119 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:38.035525 (+ 406us) server_negotiation.cc:407] Connection header received
0807 00:06:38.035778 (+ 253us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:38.035781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:38.035959 (+ 178us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:38.036106 (+ 147us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:38.036277 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.036898 (+ 621us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.037789 (+ 891us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:38.038140 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:38.038239 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:38.038654 (+ 415us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:38.039038 (+ 384us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:38.039146 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:38.039887 (+ 741us) server_negotiation.cc:299] Negotiation successful
0807 00:06:38.039954 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":80,"threads_started":1}
I20260807 00:06:38.049031 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2193
I20260807 00:06:38.065680 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2330
I20260807 00:06:38.083815 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2450
2026-08-07T00:06:38Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (3415 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Aug 07 00:06:38 dist-test-slave-pww0 krb5kdc[2799](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Aug 07 00:06:38 dist-test-slave-pww0 krb5kdc[2799](info): set up 2 sockets
Aug 07 00:06:38 dist-test-slave-pww0 krb5kdc[2799](info): commencing operation
krb5kdc: starting...
W20260807 00:06:39.263867 31537 mini_kdc.cc:121] Time spent starting KDC: real 1.149s user 0.004s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/test-user.keytab.
Aug 07 00:06:39 dist-test-slave-pww0 krb5kdc[2799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-08-07T00:06:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-08-07T00:06:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.30.204.126@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.126@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
Entry for principal kudu/127.30.204.126 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab.
I20260807 00:06:39.394516 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:45209
--webserver_interface=127.30.204.126
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:37475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:45209,127.30.204.125:33131,127.30.204.124:37455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260807 00:06:39.512180 2824 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:06:39.512517 2824 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:39.512565 2824 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:39.512633 2824 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:39.516960 2824 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:39.517081 2824 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:39.517113 2824 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:39.517138 2824 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:39.517175 2824 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:39.522934 2824 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:37475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:45209,127.30.204.125:33131,127.30.204.124:37455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.126.keytab
--principal=kudu/127.30.204.126
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.126:45209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.30.204.126
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2824
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:39.524194 2824 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:39.525331 2824 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:39.534397 2841 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:39.534390 2838 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:39.534390 2839 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:39.534493 2824 server_base.cc:1034] running on GCE node
I20260807 00:06:39.535305 2824 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:39.536592 2824 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:39.537786 2824 hybrid_clock.cc:648] HybridClock initialized: now 1786061199537766 us; error 37 us; skew 500 ppm
Aug 07 00:06:39 dist-test-slave-pww0 krb5kdc[2799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:39.540997 2824 init.cc:377] Logged in from keytab as kudu/127.30.204.126@KRBTEST.COM (short username kudu)
I20260807 00:06:39.542047 2824 webserver.cc:460] Webserver started at http://127.30.204.126:42791/ using document root <none> and password file <none>
I20260807 00:06:39.542605 2824 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:39.542676 2824 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:39.542908 2824 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:39.545274 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data/instance:
uuid: "b4b2741156844beaa6e06cb39d13d4f6"
format_stamp: "Formatted at 2026-08-07 00:06:39 on dist-test-slave-pww0"
server_key: "89d7a1804e8cb074ab56af1c63122100"
server_key_iv: "2438cbaca8c56f0489637446e6e5c08e"
server_key_version: "encryptionkey@0"
I20260807 00:06:39.545881 2824 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/wal/instance:
uuid: "b4b2741156844beaa6e06cb39d13d4f6"
format_stamp: "Formatted at 2026-08-07 00:06:39 on dist-test-slave-pww0"
server_key: "89d7a1804e8cb074ab56af1c63122100"
server_key_iv: "2438cbaca8c56f0489637446e6e5c08e"
server_key_version: "encryptionkey@0"
I20260807 00:06:39.549861 2824 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.001s
I20260807 00:06:39.552531 2850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:39.553817 2824 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.003s sys 0.000s
I20260807 00:06:39.553941 2824 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/wal
uuid: "b4b2741156844beaa6e06cb39d13d4f6"
format_stamp: "Formatted at 2026-08-07 00:06:39 on dist-test-slave-pww0"
server_key: "89d7a1804e8cb074ab56af1c63122100"
server_key_iv: "2438cbaca8c56f0489637446e6e5c08e"
server_key_version: "encryptionkey@0"
I20260807 00:06:39.554044 2824 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:39.602967 2824 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:39.606081 2824 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:39.606287 2824 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:39.613713 2824 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.126:45209
I20260807 00:06:39.613982 2932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.126:45209 every 8 connection(s)
I20260807 00:06:39.614833 2824 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/data/info.pb
I20260807 00:06:39.617823 2933 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:39.621842 2933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:39.622754 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2824
I20260807 00:06:39.622888 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-0/wal/instance
I20260807 00:06:39.623150 31537 external_mini_cluster.cc:1428] Setting key a3fd8baa64a69a5e817c853649380b2a
WARNING: no policy specified for kudu/127.30.204.125@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.125@KRBTEST.COM" created.
Aug 07 00:06:39 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:39.640986 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.624275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:45209 (local address 127.0.0.1:53626)
0807 00:06:39.624895 (+ 620us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.624967 (+ 72us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:39.626097 (+ 1130us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:39.626932 (+ 835us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:39.626938 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:39.627404 (+ 466us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:39.627720 (+ 316us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.627737 (+ 17us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.628892 (+ 1155us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.628895 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.629668 (+ 773us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.629678 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.630355 (+ 677us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.630359 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.630474 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.631050 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:39.631070 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:39.635395 (+ 4325us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:39.637673 (+ 2278us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.637685 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.637699 (+ 14us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.638042 (+ 343us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.638482 (+ 440us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.638485 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.638487 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.638627 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.639394 (+ 767us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:39.639400 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:39.639582 (+ 182us) client_negotiation.cc:769] Sending connection context
0807 00:06:39.639741 (+ 159us) client_negotiation.cc:240] Negotiation successful
0807 00:06:39.640045 (+ 304us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":348,"thread_start_us":131,"threads_started":1}
I20260807 00:06:39.640988 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.625119 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53626 (local address 127.30.204.126:45209)
0807 00:06:39.625449 (+ 330us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:39.625462 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:39.625522 (+ 60us) server_negotiation.cc:407] Connection header received
0807 00:06:39.626482 (+ 960us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:39.626486 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:39.626727 (+ 241us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:39.626910 (+ 183us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:39.627874 (+ 964us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.628765 (+ 891us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.629788 (+ 1023us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.630184 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.630358 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.635656 (+ 5298us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:39.635666 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:39.635684 (+ 18us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:39.635719 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:39.637525 (+ 1806us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.638185 (+ 660us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.638192 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.638194 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.638366 (+ 172us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.638723 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.638726 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.638728 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.639282 (+ 554us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:39.639418 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:39.639865 (+ 447us) server_negotiation.cc:299] Negotiation successful
0807 00:06:39.640185 (+ 320us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":80,"server-negotiator.queue_time_us":221,"thread_start_us":87,"threads_started":1}
I20260807 00:06:39.643707 2933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:39.644537 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.644256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42844)
0807 00:06:39.644386 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.644465 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":50}
W20260807 00:06:39.644767 2852 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
I20260807 00:06:39.645742 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.645455 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42850)
0807 00:06:39.645613 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.645676 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260807 00:06:39.645941 2933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:33131: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.125 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab.
Entry for principal kudu/127.30.204.125 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab.
I20260807 00:06:39.654968 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:33131
--webserver_interface=127.30.204.125
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab
--principal=kudu/127.30.204.125
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:37475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:45209,127.30.204.125:33131,127.30.204.124:37455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260807 00:06:39.695232 2933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } attempt: 1
I20260807 00:06:39.696276 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.695927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42856)
0807 00:06:39.696118 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.696191 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260807 00:06:39.697026 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.696802 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42870)
0807 00:06:39.696897 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.696945 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260807 00:06:39.697165 2933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:33131: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
I20260807 00:06:39.765354 2933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } attempt: 2
I20260807 00:06:39.766359 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.765996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42884)
0807 00:06:39.766200 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.766274 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87}
I20260807 00:06:39.767259 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.766991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42892)
0807 00:06:39.767129 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.767190 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64}
W20260807 00:06:39.767421 2933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.125:33131: Network error: Client connection negotiation failed: client connection to 127.30.204.125:33131: connect: Connection refused (error 111)
W20260807 00:06:39.776126 2944 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:06:39.776446 2944 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:39.776474 2944 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:39.776537 2944 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260807 00:06:39.780753 2944 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:39.780838 2944 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:39.780857 2944 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:39.780874 2944 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:39.780887 2944 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:39.786677 2944 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:37475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:45209,127.30.204.125:33131,127.30.204.124:37455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.125.keytab
--principal=kudu/127.30.204.125
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.125:33131
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.30.204.125
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2944
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:39.787986 2944 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:39.789079 2944 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:39.798017 2952 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:39.798236 2949 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:39.798312 2950 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:39.798311 2944 server_base.cc:1034] running on GCE node
I20260807 00:06:39.798998 2944 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:39.800303 2944 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:39.801482 2944 hybrid_clock.cc:648] HybridClock initialized: now 1786061199801467 us; error 31 us; skew 500 ppm
Aug 07 00:06:39 dist-test-slave-pww0 krb5kdc[2799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:39.804414 2944 init.cc:377] Logged in from keytab as kudu/127.30.204.125@KRBTEST.COM (short username kudu)
I20260807 00:06:39.805456 2944 webserver.cc:460] Webserver started at http://127.30.204.125:45249/ using document root <none> and password file <none>
I20260807 00:06:39.806015 2944 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:39.806075 2944 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:39.806262 2944 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:39.808516 2944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data/instance:
uuid: "d14b07121e174defb736cddd1626ac0b"
format_stamp: "Formatted at 2026-08-07 00:06:39 on dist-test-slave-pww0"
server_key: "ec3b51cf25b699c57174c86ecd300abc"
server_key_iv: "aa77c8e2ec0538953ec0af07849cbf47"
server_key_version: "encryptionkey@0"
I20260807 00:06:39.809067 2944 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/wal/instance:
uuid: "d14b07121e174defb736cddd1626ac0b"
format_stamp: "Formatted at 2026-08-07 00:06:39 on dist-test-slave-pww0"
server_key: "ec3b51cf25b699c57174c86ecd300abc"
server_key_iv: "aa77c8e2ec0538953ec0af07849cbf47"
server_key_version: "encryptionkey@0"
I20260807 00:06:39.813104 2944 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.004s sys 0.000s
I20260807 00:06:39.815850 2965 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:39.817288 2944 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.001s
I20260807 00:06:39.817453 2944 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/wal
uuid: "d14b07121e174defb736cddd1626ac0b"
format_stamp: "Formatted at 2026-08-07 00:06:39 on dist-test-slave-pww0"
server_key: "ec3b51cf25b699c57174c86ecd300abc"
server_key_iv: "aa77c8e2ec0538953ec0af07849cbf47"
server_key_version: "encryptionkey@0"
I20260807 00:06:39.817580 2944 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:39.829764 2944 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:39.833642 2944 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:39.833886 2944 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:39.842005 2944 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.125:33131
I20260807 00:06:39.842342 3046 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.125:33131 every 8 connection(s)
I20260807 00:06:39.843133 2944 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/data/info.pb
I20260807 00:06:39.843432 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 2944
I20260807 00:06:39.843590 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-1/wal/instance
I20260807 00:06:39.843954 31537 external_mini_cluster.cc:1428] Setting key c6117be50f9cb3ef5b5ee244e71a2096
I20260807 00:06:39.846832 3048 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:39.851037 3048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.30.204.124@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.30.204.124@KRBTEST.COM" created.
I20260807 00:06:39.858595 2933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } attempt: 3
Aug 07 00:06:39 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
Aug 07 00:06:39 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:06:39.868373 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.853225 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53636 (local address 127.30.204.126:45209)
0807 00:06:39.853392 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:39.853397 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:39.854462 (+ 1065us) server_negotiation.cc:407] Connection header received
0807 00:06:39.855578 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:39.855581 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:39.855645 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:39.855738 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:39.857000 (+ 1262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.857771 (+ 771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.859977 (+ 2206us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.860301 (+ 324us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.860397 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.863250 (+ 2853us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:39.863254 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:39.863257 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:39.863286 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:39.865892 (+ 2606us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.866467 (+ 575us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.866471 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.866473 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.866568 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.866914 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.866918 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.866919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.867176 (+ 257us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:39.867326 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:39.868109 (+ 783us) server_negotiation.cc:299] Negotiation successful
0807 00:06:39.868242 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:06:39.869069 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.859093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42906)
0807 00:06:39.859285 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.859296 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:39.859366 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:39.860077 (+ 711us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:39.860081 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:39.860279 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:39.860398 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.860404 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.861755 (+ 1351us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.861758 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.862333 (+ 575us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.862345 (+ 12us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.862985 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.862989 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.863097 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.863636 (+ 539us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:39.863648 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:39.865015 (+ 1367us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:39.867296 (+ 2281us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.867299 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.867301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.867591 (+ 290us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.867987 (+ 396us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.867990 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.867992 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.868090 (+ 98us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.868602 (+ 512us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:39.868605 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:39.868716 (+ 111us) client_negotiation.cc:769] Sending connection context
0807 00:06:39.868797 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:06:39.868929 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120}
I20260807 00:06:39.869587 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.853736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:45209 (local address 127.0.0.1:53636)
0807 00:06:39.854291 (+ 555us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.854340 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:39.855245 (+ 905us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:39.855920 (+ 675us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:39.855928 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:39.856311 (+ 383us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:39.856831 (+ 520us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.856846 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.857914 (+ 1068us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.857917 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.859842 (+ 1925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.859855 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.860453 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.860458 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.860637 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.861207 (+ 570us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:39.861224 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:39.863097 (+ 1873us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:39.866016 (+ 2919us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.866037 (+ 21us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.866050 (+ 13us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.866363 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.866663 (+ 300us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.866665 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.866666 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.866826 (+ 160us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.867343 (+ 517us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:39.867351 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:39.867778 (+ 427us) client_negotiation.cc:769] Sending connection context
0807 00:06:39.868032 (+ 254us) client_negotiation.cc:240] Negotiation successful
0807 00:06:39.868423 (+ 391us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":325,"thread_start_us":144,"threads_started":1}
I20260807 00:06:39.869753 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.859206 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42906 (local address 127.30.204.125:33131)
0807 00:06:39.859618 (+ 412us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:39.859629 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:39.859651 (+ 22us) server_negotiation.cc:407] Connection header received
0807 00:06:39.859688 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:39.859693 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:39.859867 (+ 174us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:39.860081 (+ 214us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:39.860527 (+ 446us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.861623 (+ 1096us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.862452 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.862866 (+ 414us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.862965 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.865199 (+ 2234us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:39.865208 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:39.865228 (+ 20us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:39.865258 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:39.867171 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.867730 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.867736 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.867738 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.867883 (+ 145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.868191 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.868194 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.868195 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.868479 (+ 284us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:39.869293 (+ 814us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:39.869401 (+ 108us) server_negotiation.cc:299] Negotiation successful
0807 00:06:39.869557 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":308,"spinlock_wait_cycles":1920,"thread_start_us":162,"threads_started":1}
I20260807 00:06:39.870988 3048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:39.872004 2933 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:39.872776 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.872487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42400)
0807 00:06:39.872648 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.872702 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":85}
I20260807 00:06:39.873701 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.873372 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42408)
0807 00:06:39.873570 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.873627 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
W20260807 00:06:39.873914 2933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37455: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.30.204.124 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab.
Entry for principal kudu/127.30.204.124 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab.
I20260807 00:06:39.875041 31537 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
/tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:37455
--webserver_interface=127.30.204.124
--webserver_port=0
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab
--principal=kudu/127.30.204.124
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.30.204.84:37475
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.30.204.126:45209,127.30.204.125:33131,127.30.204.124:37455
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Aug 07 00:06:39 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:06:39.880254 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.871588 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42912)
0807 00:06:39.871697 (+ 109us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.871712 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:39.871794 (+ 82us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:39.872138 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:39.872141 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:39.872319 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:39.872446 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.872453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.873476 (+ 1023us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.873479 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.874222 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:39.874232 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.874738 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.874742 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:39.874843 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.875337 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:39.875351 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:39.876745 (+ 1394us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:39.878363 (+ 1618us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.878367 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.878369 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.878570 (+ 201us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.878947 (+ 377us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:39.878950 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:39.878952 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:39.879045 (+ 93us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:39.879658 (+ 613us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:39.879662 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:39.879796 (+ 134us) client_negotiation.cc:769] Sending connection context
0807 00:06:39.879889 (+ 93us) client_negotiation.cc:240] Negotiation successful
0807 00:06:39.880063 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260807 00:06:39.880275 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.871638 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42912 (local address 127.30.204.125:33131)
0807 00:06:39.871755 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:39.871759 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:39.871777 (+ 18us) server_negotiation.cc:407] Connection header received
0807 00:06:39.871954 (+ 177us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:39.871958 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:39.872013 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:39.872098 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:39.872566 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.873360 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.874340 (+ 980us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:39.874624 (+ 284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:39.874712 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:39.876883 (+ 2171us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:39.876887 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:39.876890 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:39.876914 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:39.878251 (+ 1337us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.878656 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.878660 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.878662 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.878823 (+ 161us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:39.879131 (+ 308us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:39.879134 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:39.879136 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:39.879551 (+ 415us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:39.879652 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:39.879961 (+ 309us) server_negotiation.cc:299] Negotiation successful
0807 00:06:39.880130 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":53,"server-negotiator.queue_time_us":53}
I20260807 00:06:39.881443 3048 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:39.882169 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.881894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42422)
0807 00:06:39.882011 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.882102 (+ 91us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260807 00:06:39.882714 2968 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
I20260807 00:06:39.883708 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.883503 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42432)
0807 00:06:39.883603 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.883652 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260807 00:06:39.883859 3048 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37455: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
I20260807 00:06:39.905097 2933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } attempt: 1
I20260807 00:06:39.906075 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.905693 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42434)
0807 00:06:39.905884 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.905966 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 00:06:39.906937 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.906689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42444)
0807 00:06:39.906803 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.906862 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 00:06:39.907096 2933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37455: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
I20260807 00:06:39.933153 3048 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } attempt: 1
I20260807 00:06:39.934100 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.933773 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42456)
0807 00:06:39.933932 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.934006 (+ 74us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":61}
I20260807 00:06:39.935050 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.934776 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42468)
0807 00:06:39.934909 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.934977 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":53}
W20260807 00:06:39.935202 3048 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37455: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
I20260807 00:06:39.982285 2933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } attempt: 2
I20260807 00:06:39.983258 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.982910 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42476)
0807 00:06:39.983086 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.983157 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":67}
I20260807 00:06:39.984048 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:39.983839 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42490)
0807 00:06:39.983938 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:39.983987 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260807 00:06:39.984220 2933 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37455: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
W20260807 00:06:40.000859 3056 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260807 00:06:40.001181 3056 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260807 00:06:40.001206 3056 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260807 00:06:40.001252 3056 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260807 00:06:40.003382 3048 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } attempt: 2
I20260807 00:06:40.004315 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.003991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42492)
0807 00:06:40.004162 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:40.004232 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260807 00:06:40.005043 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.004823 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42504)
0807 00:06:40.004931 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:40.004984 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260807 00:06:40.005157 3048 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.30.204.124:37455: Network error: Client connection negotiation failed: client connection to 127.30.204.124:37455: connect: Connection refused (error 111)
W20260807 00:06:40.005854 3056 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260807 00:06:40.005913 3056 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260807 00:06:40.005941 3056 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260807 00:06:40.005972 3056 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260807 00:06:40.006001 3056 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260807 00:06:40.011852 3056 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.30.204.84:37475
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.30.204.126:45209,127.30.204.125:33131,127.30.204.124:37455
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/krb5kdc/kudu_127.30.204.124.keytab
--principal=kudu/127.30.204.124
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.30.204.124:37455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.30.204.124
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3056
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type DEBUG
built by systest at 06 Aug 2026 23:05:08 UTC on ccycloud.quasar-eswqwf.root.comops.site
build id 4571539
I20260807 00:06:40.013105 3056 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260807 00:06:40.014261 3056 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260807 00:06:40.023839 3066 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:40.023826 3063 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260807 00:06:40.023826 3064 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260807 00:06:40.023887 3056 server_base.cc:1034] running on GCE node
I20260807 00:06:40.024667 3056 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260807 00:06:40.025885 3056 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260807 00:06:40.027065 3056 hybrid_clock.cc:648] HybridClock initialized: now 1786061200027047 us; error 33 us; skew 500 ppm
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061200, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260807 00:06:40.030113 3056 init.cc:377] Logged in from keytab as kudu/127.30.204.124@KRBTEST.COM (short username kudu)
I20260807 00:06:40.031181 3056 webserver.cc:460] Webserver started at http://127.30.204.124:33793/ using document root <none> and password file <none>
I20260807 00:06:40.031838 3056 fs_manager.cc:359] Metadata directory not provided
I20260807 00:06:40.031916 3056 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260807 00:06:40.032148 3056 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260807 00:06:40.034379 3056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data/instance:
uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0"
format_stamp: "Formatted at 2026-08-07 00:06:40 on dist-test-slave-pww0"
server_key: "4f8f22d442fbee132dba99d5cfc4481b"
server_key_iv: "b6dd5ba28ba491d5d38b5ef9a67a9e28"
server_key_version: "encryptionkey@0"
I20260807 00:06:40.034960 3056 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/wal/instance:
uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0"
format_stamp: "Formatted at 2026-08-07 00:06:40 on dist-test-slave-pww0"
server_key: "4f8f22d442fbee132dba99d5cfc4481b"
server_key_iv: "b6dd5ba28ba491d5d38b5ef9a67a9e28"
server_key_version: "encryptionkey@0"
I20260807 00:06:40.039110 3056 fs_manager.cc:691] Time spent creating directory manager: real 0.004s user 0.003s sys 0.004s
I20260807 00:06:40.042032 3074 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260807 00:06:40.043390 3056 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.002s sys 0.000s
I20260807 00:06:40.043566 3056 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data,/tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/wal
uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0"
format_stamp: "Formatted at 2026-08-07 00:06:40 on dist-test-slave-pww0"
server_key: "4f8f22d442fbee132dba99d5cfc4481b"
server_key_iv: "b6dd5ba28ba491d5d38b5ef9a67a9e28"
server_key_version: "encryptionkey@0"
I20260807 00:06:40.043692 3056 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260807 00:06:40.068606 3056 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260807 00:06:40.072043 3056 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260807 00:06:40.072268 3056 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260807 00:06:40.079687 3056 rpc_server.cc:307] RPC server started. Bound to: 127.30.204.124:37455
I20260807 00:06:40.079962 3155 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.30.204.124:37455 every 8 connection(s)
I20260807 00:06:40.080760 3056 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/data/info.pb
I20260807 00:06:40.083401 2933 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } attempt: 3
I20260807 00:06:40.083668 31537 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu as pid 3056
I20260807 00:06:40.083818 31537 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskl_Zz3S/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1786060901439409-31537-0/minicluster-data/master-2/wal/instance
I20260807 00:06:40.083873 3156 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260807 00:06:40.084142 31537 external_mini_cluster.cc:1428] Setting key 65a508fe68d1c4390790b3ffe5ee6231
I20260807 00:06:40.089336 3156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } has no permanent_uuid. Determining permanent_uuid...
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.126@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
I20260807 00:06:40.096349 3048 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } attempt: 3
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061200, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:40.098604 2936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.084104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42508)
0807 00:06:40.084262 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:40.084280 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:40.084384 (+ 104us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:40.087103 (+ 2719us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:40.087106 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:40.087346 (+ 240us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:40.087506 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.087516 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.089325 (+ 1809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.089329 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.090104 (+ 775us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.090119 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.091206 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.091211 (+ 5us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.091301 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.091884 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:40.091900 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:40.093430 (+ 1530us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:40.096014 (+ 2584us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.096017 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.096020 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.096312 (+ 292us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.096822 (+ 510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.096825 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.096827 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.096927 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.098074 (+ 1147us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:40.098077 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:40.098182 (+ 105us) client_negotiation.cc:769] Sending connection context
0807 00:06:40.098263 (+ 81us) client_negotiation.cc:240] Negotiation successful
0807 00:06:40.098422 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 00:06:40.100288 3160 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.085181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42520 (local address 127.30.204.124:37455)
0807 00:06:40.085683 (+ 502us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.085695 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.085731 (+ 36us) server_negotiation.cc:407] Connection header received
0807 00:06:40.086436 (+ 705us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.086442 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.086807 (+ 365us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.087127 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.088207 (+ 1080us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.089156 (+ 949us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.090171 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.091063 (+ 892us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.091254 (+ 191us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.095001 (+ 3747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.095004 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.095006 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.095030 (+ 24us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.096877 (+ 1847us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.097415 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.097419 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.097420 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.097503 (+ 83us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.097833 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.097837 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.097838 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.098022 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.098199 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.098704 (+ 505us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.099045 (+ 341us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":468,"server-negotiator.queue_time_us":430,"spinlock_wait_cycles":35072,"thread_start_us":144,"threads_started":1}
I20260807 00:06:40.100288 3161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.085090 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42508 (local address 127.30.204.124:37455)
0807 00:06:40.085677 (+ 587us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.085693 (+ 16us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.085730 (+ 37us) server_negotiation.cc:407] Connection header received
0807 00:06:40.086408 (+ 678us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.086426 (+ 18us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.086804 (+ 378us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.087125 (+ 321us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.087719 (+ 594us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.089156 (+ 1437us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.090216 (+ 1060us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.091063 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.091258 (+ 195us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.093615 (+ 2357us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.093621 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.093635 (+ 14us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.093669 (+ 34us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.095910 (+ 2241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.096446 (+ 536us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.096453 (+ 7us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.096459 (+ 6us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.096701 (+ 242us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.097039 (+ 338us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.097043 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.097044 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.097951 (+ 907us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.099080 (+ 1129us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.099151 (+ 71us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.099254 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":181,"server-negotiator.queue_time_us":452,"spinlock_wait_cycles":85504,"thread_start_us":186,"threads_started":1}
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.125@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
I20260807 00:06:40.102864 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.091692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53642 (local address 127.30.204.126:45209)
0807 00:06:40.091826 (+ 134us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.091831 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.092202 (+ 371us) server_negotiation.cc:407] Connection header received
0807 00:06:40.092325 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.092328 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.092374 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.092436 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.093283 (+ 847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.093960 (+ 677us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.095159 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.095500 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.095605 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.099108 (+ 3503us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.099112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.099115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.099146 (+ 31us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.100788 (+ 1642us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.101319 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.101323 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.101324 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.101419 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.101751 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.101754 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.101756 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.102063 (+ 307us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.102138 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.102566 (+ 428us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.102697 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260807 00:06:40.103353 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.091366 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.126:45209 (local address 127.0.0.1:53642)
0807 00:06:40.092069 (+ 703us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:40.092111 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:40.092239 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:40.092503 (+ 264us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:40.092510 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:40.092972 (+ 462us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:40.093115 (+ 143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.093123 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.094067 (+ 944us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.094071 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.095026 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.095039 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.095620 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.095624 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.095746 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.096388 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:40.096403 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:40.098958 (+ 2555us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:40.100887 (+ 1929us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.100896 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.100902 (+ 6us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.101198 (+ 296us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.101534 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.101537 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.101539 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.101633 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.102213 (+ 580us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:40.102219 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:40.102338 (+ 119us) client_negotiation.cc:769] Sending connection context
0807 00:06:40.102998 (+ 660us) client_negotiation.cc:240] Negotiation successful
0807 00:06:40.103189 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":548,"thread_start_us":176,"threads_started":1}
I20260807 00:06:40.105005 3156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:40.106089 2933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6: Bootstrap starting.
I20260807 00:06:40.107836 3051 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.096837 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42526)
0807 00:06:40.097061 (+ 224us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:40.097075 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:40.097156 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:40.097981 (+ 825us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:40.097985 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:40.098183 (+ 198us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:40.098304 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.098312 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.099391 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.099395 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.100226 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.100241 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.100965 (+ 724us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.100969 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.101061 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.101556 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:40.101571 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:40.103111 (+ 1540us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:40.106041 (+ 2930us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.106044 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.106046 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.106296 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.106699 (+ 403us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.106703 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.106704 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.106807 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.107251 (+ 444us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:40.107254 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:40.107357 (+ 103us) client_negotiation.cc:769] Sending connection context
0807 00:06:40.107439 (+ 82us) client_negotiation.cc:240] Negotiation successful
0807 00:06:40.107634 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139}
I20260807 00:06:40.108956 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.097364 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42526 (local address 127.30.204.124:37455)
0807 00:06:40.097717 (+ 353us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.097726 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.097740 (+ 14us) server_negotiation.cc:407] Connection header received
0807 00:06:40.097774 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.097777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.097820 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.098423 (+ 603us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.098536 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.099255 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.100381 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.100847 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.100950 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.103685 (+ 2735us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.103689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.103692 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.103718 (+ 26us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.105407 (+ 1689us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.106488 (+ 1081us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.106491 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.106492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.106579 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.106922 (+ 343us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.106925 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.106927 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.107139 (+ 212us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.108556 (+ 1417us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.108639 (+ 83us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.108785 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":274,"thread_start_us":130,"threads_started":1}
I20260807 00:06:40.109689 2933 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6: Neither blocks nor log segments found. Creating new log.
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061200, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
I20260807 00:06:40.110808 2933 log.cc:824] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6: Log is configured to *not* fsync() on all Append() calls
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.126@KRBTEST.COM
I20260807 00:06:40.113000 3048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: Bootstrap starting.
I20260807 00:06:40.113359 2933 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6: No bootstrap required, opened a new log
I20260807 00:06:40.115237 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.105594 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42918 (local address 127.30.204.125:33131)
0807 00:06:40.105729 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.105734 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.105747 (+ 13us) server_negotiation.cc:407] Connection header received
0807 00:06:40.105784 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.105786 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.105829 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.105909 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.106353 (+ 444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.107059 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.108177 (+ 1118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.108525 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.108590 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.111521 (+ 2931us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.111526 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.111529 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.111558 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.113296 (+ 1738us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.113803 (+ 507us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.113807 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.113808 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.113890 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.114204 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.114207 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.114209 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.114494 (+ 285us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.114568 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.114941 (+ 373us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.115074 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":66}
I20260807 00:06:40.115700 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.103659 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53652 (local address 127.30.204.126:45209)
0807 00:06:40.104401 (+ 742us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.104405 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.104417 (+ 12us) server_negotiation.cc:407] Connection header received
0807 00:06:40.104447 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.104449 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.104715 (+ 266us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.104788 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.106071 (+ 1283us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.106967 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.108177 (+ 1210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.108516 (+ 339us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.108591 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.111588 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.111591 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.111594 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.111621 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.113613 (+ 1992us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.114184 (+ 571us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.114187 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.114189 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.114290 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.114602 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.114606 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.114608 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.114806 (+ 198us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.114950 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.115383 (+ 433us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.115548 (+ 165us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":672}
I20260807 00:06:40.115739 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.105482 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.125:33131 (local address 127.0.0.1:42918)
0807 00:06:40.105622 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:40.105633 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:40.105689 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:40.105965 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:40.105968 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:40.106128 (+ 160us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:40.106220 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.106227 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.107182 (+ 955us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.107185 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.108036 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.108046 (+ 10us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.108756 (+ 710us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.108760 (+ 4us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.108849 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.109364 (+ 515us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:40.109379 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:40.111300 (+ 1921us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:40.113436 (+ 2136us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.113440 (+ 4us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.113443 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.113676 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.114008 (+ 332us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.114011 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.114013 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.114105 (+ 92us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.114637 (+ 532us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:40.114640 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:40.114752 (+ 112us) client_negotiation.cc:769] Sending connection context
0807 00:06:40.115391 (+ 639us) client_negotiation.cc:240] Negotiation successful
0807 00:06:40.115581 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":89}
I20260807 00:06:40.116515 3156 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } has no permanent_uuid. Determining permanent_uuid...
I20260807 00:06:40.116787 3048 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:40.117792 3048 log.cc:824] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:40.118013 2933 raft_consensus.cc:348] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.118281 2933 raft_consensus.cc:729] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b4b2741156844beaa6e06cb39d13d4f6, State: Initialized, Role: FOLLOWER
I20260807 00:06:40.118657 2933 consensus_queue.cc:260] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.120280 2933 sys_catalog.cc:564] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:40.120676 3048 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: No bootstrap required, opened a new log
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061200, etypes {rep=17 tkt=17 ses=17}, kudu/127.30.204.124@KRBTEST.COM for kudu/127.30.204.124@KRBTEST.COM
Aug 07 00:06:40 dist-test-slave-pww0 krb5kdc[2799](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1786061199, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.30.204.125@KRBTEST.COM
W20260807 00:06:40.126447 3187 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:40.126554 3187 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:40.127679 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.117155 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42528 (local address 127.30.204.124:37455)
0807 00:06:40.118391 (+ 1236us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.118395 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.118410 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:06:40.118444 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.118447 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.118490 (+ 43us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.118572 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.118987 (+ 415us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.119702 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.120777 (+ 1075us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.121083 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.121198 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.124246 (+ 3048us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.124250 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.124252 (+ 2us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.124277 (+ 25us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.125879 (+ 1602us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.126301 (+ 422us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.126304 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.126305 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.126370 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.126633 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.126636 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.126639 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.126889 (+ 250us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.126964 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.127294 (+ 330us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.127439 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1151}
I20260807 00:06:40.128098 3162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.117108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.30.204.124:37455 (local address 127.0.0.1:42528)
0807 00:06:40.117235 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0807 00:06:40.117250 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0807 00:06:40.117326 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0807 00:06:40.118606 (+ 1280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0807 00:06:40.118609 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0807 00:06:40.118788 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0807 00:06:40.118879 (+ 91us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.118885 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.119822 (+ 937us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.119825 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.120665 (+ 840us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0807 00:06:40.120676 (+ 11us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.121332 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.121335 (+ 3us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0807 00:06:40.121405 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.121882 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0807 00:06:40.121898 (+ 16us) client_negotiation.cc:656] calling sasl_client_start()
0807 00:06:40.124089 (+ 2191us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0807 00:06:40.125992 (+ 1903us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.125995 (+ 3us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.125997 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.126216 (+ 219us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.126464 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0807 00:06:40.126466 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0807 00:06:40.126469 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0807 00:06:40.126548 (+ 79us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0807 00:06:40.127023 (+ 475us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0807 00:06:40.127025 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0807 00:06:40.127113 (+ 88us) client_negotiation.cc:769] Sending connection context
0807 00:06:40.127828 (+ 715us) client_negotiation.cc:240] Negotiation successful
0807 00:06:40.127962 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":53}
I20260807 00:06:40.126462 2824 master_runner.cc:186] Error getting master registration for 127.30.204.125:33131: OK, instance_id { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" instance_seqno: 1786061199839748 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:06:40.129863 2824 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:06:40.129951 3048 raft_consensus.cc:348] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.130231 3048 raft_consensus.cc:729] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d14b07121e174defb736cddd1626ac0b, State: Initialized, Role: FOLLOWER
I20260807 00:06:40.130851 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.117623 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42920 (local address 127.30.204.125:33131)
0807 00:06:40.117726 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.117729 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.117738 (+ 9us) server_negotiation.cc:407] Connection header received
0807 00:06:40.117767 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.117769 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.117802 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.117863 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.118495 (+ 632us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.119216 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.120481 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.120812 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.120879 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.126238 (+ 5359us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.126243 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.126246 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.126276 (+ 30us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.127987 (+ 1711us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.129004 (+ 1017us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.129007 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.129009 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.129106 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.129443 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.129447 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.129448 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.129658 (+ 210us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.129831 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.130515 (+ 684us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.130640 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260807 00:06:40.130635 3048 consensus_queue.cc:260] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.131582 3156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: Bootstrap starting.
I20260807 00:06:40.132725 3048 sys_catalog.cc:564] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:40.134578 31537 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260807 00:06:40.134593 3156 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: Neither blocks nor log segments found. Creating new log.
I20260807 00:06:40.135533 3156 log.cc:824] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: Log is configured to *not* fsync() on all Append() calls
I20260807 00:06:40.136626 2894 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "b4b2741156844beaa6e06cb39d13d4f6"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:53652
I20260807 00:06:40.136880 2894 raft_consensus.cc:482] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260807 00:06:40.137009 2894 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [term 0 FOLLOWER]: Advancing to term 1
W20260807 00:06:40.138780 3206 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:40.138870 3206 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:40.138868 2894 raft_consensus.cc:504] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.139066 3190 sys_catalog.cc:455] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.139195 3190 sys_catalog.cc:458] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:40.139495 3169 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.139637 3169 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:40.140004 2894 leader_election.cc:288] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [CANDIDATE]: Term 1 election: Requested vote from peers d14b07121e174defb736cddd1626ac0b (127.30.204.125:33131), 6ee2f5fe5e92432480c16a6c83c2f7c0 (127.30.204.124:37455)
I20260807 00:06:40.141000 3008 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4b2741156844beaa6e06cb39d13d4f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "d14b07121e174defb736cddd1626ac0b"
I20260807 00:06:40.141232 3008 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:40.143146 3008 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b2741156844beaa6e06cb39d13d4f6 in term 1.
I20260807 00:06:40.143618 2852 leader_election.cc:302] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b4b2741156844beaa6e06cb39d13d4f6, d14b07121e174defb736cddd1626ac0b; no voters:
I20260807 00:06:40.144093 3169 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [term 1 FOLLOWER]: Leader election won for term 1
I20260807 00:06:40.144356 3169 raft_consensus.cc:686] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [term 1 LEADER]: Becoming Leader. State: Replica: b4b2741156844beaa6e06cb39d13d4f6, State: Running, Role: LEADER
I20260807 00:06:40.144742 3169 consensus_queue.cc:237] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.147043 3218 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4b2741156844beaa6e06cb39d13d4f6. Latest consensus state: current_term: 1 leader_uuid: "b4b2741156844beaa6e06cb39d13d4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.149395 2944 master_runner.cc:186] Error getting master registration for 127.30.204.124:37455: OK, instance_id { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" instance_seqno: 1786061200077696 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260807 00:06:40.152388 2944 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260807 00:06:40.154593 3156 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: No bootstrap required, opened a new log
I20260807 00:06:40.147171 3218 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:40.160077 3055 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.143998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42932 (local address 127.30.204.125:33131)
0807 00:06:40.146979 (+ 2981us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.146984 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.147003 (+ 19us) server_negotiation.cc:407] Connection header received
0807 00:06:40.147049 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.147052 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.147105 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.147218 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.148611 (+ 1393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.149312 (+ 701us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.153523 (+ 4211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.154126 (+ 603us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.154210 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.156108 (+ 1898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.156112 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.156115 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.156148 (+ 33us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.158004 (+ 1856us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.158556 (+ 552us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.158559 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.158561 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.158659 (+ 98us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.158969 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.158973 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.158974 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.159210 (+ 236us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.159356 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.159775 (+ 419us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.159918 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2885}
I20260807 00:06:40.162742 2938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.143386 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53654 (local address 127.30.204.126:45209)
0807 00:06:40.143607 (+ 221us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.143612 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.149038 (+ 5426us) server_negotiation.cc:407] Connection header received
0807 00:06:40.149228 (+ 190us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.149232 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.149285 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.149369 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.150824 (+ 1455us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.151623 (+ 799us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.152899 (+ 1276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.153250 (+ 351us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.153329 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.155325 (+ 1996us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.155330 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.155334 (+ 4us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.155369 (+ 35us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.157104 (+ 1735us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.158677 (+ 1573us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.158680 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.158682 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.158786 (+ 104us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.159071 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.159074 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.159076 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.159279 (+ 203us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.159366 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.162440 (+ 3074us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.162578 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127}
I20260807 00:06:40.162416 3156 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.162927 3156 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6ee2f5fe5e92432480c16a6c83c2f7c0, State: Initialized, Role: FOLLOWER
I20260807 00:06:40.163775 3156 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } }
I20260807 00:06:40.164536 3224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.164690 3224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:40.165884 3156 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [sys.catalog]: configured and running, proceeding with master startup.
I20260807 00:06:40.165963 3116 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b4b2741156844beaa6e06cb39d13d4f6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0"
I20260807 00:06:40.166262 3116 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [term 0 FOLLOWER]: Advancing to term 1
I20260807 00:06:40.168428 3116 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b4b2741156844beaa6e06cb39d13d4f6 in term 1.
I20260807 00:06:40.169595 3221 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260807 00:06:40.170331 3163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:40.159934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42538 (local address 127.30.204.124:37455)
0807 00:06:40.160081 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.160099 (+ 18us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.161076 (+ 977us) server_negotiation.cc:407] Connection header received
0807 00:06:40.161277 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.161280 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.161339 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.161389 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.162149 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.162855 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.164252 (+ 1397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:40.164568 (+ 316us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:40.164652 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:40.166771 (+ 2119us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0807 00:06:40.166775 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0807 00:06:40.166778 (+ 3us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0807 00:06:40.166807 (+ 29us) server_negotiation.cc:922] calling sasl_server_start()
0807 00:06:40.168453 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.168985 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.168988 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.168991 (+ 3us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.169088 (+ 97us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0807 00:06:40.169414 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0807 00:06:40.169418 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0807 00:06:40.169419 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0807 00:06:40.169613 (+ 194us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0807 00:06:40.169749 (+ 136us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:40.170035 (+ 286us) server_negotiation.cc:299] Negotiation successful
0807 00:06:40.170163 (+ 128us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260807 00:06:40.170727 3221 catalog_manager.cc:1484] Initializing Kudu cluster ID...
W20260807 00:06:40.174345 3236 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260807 00:06:40.174728 3236 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260807 00:06:40.178159 3116 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [term 1 FOLLOWER]: Refusing update from remote peer b4b2741156844beaa6e06cb39d13d4f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:40.178440 3008 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [term 1 FOLLOWER]: Refusing update from remote peer b4b2741156844beaa6e06cb39d13d4f6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260807 00:06:40.178975 3218 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:40.179704 3218 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260807 00:06:40.183542 3190 sys_catalog.cc:455] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4b2741156844beaa6e06cb39d13d4f6. Latest consensus state: current_term: 1 leader_uuid: "b4b2741156844beaa6e06cb39d13d4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.183699 3190 sys_catalog.cc:458] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:40.184836 3224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b4b2741156844beaa6e06cb39d13d4f6. Latest consensus state: current_term: 1 leader_uuid: "b4b2741156844beaa6e06cb39d13d4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.185010 3224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:40.187367 3169 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4b2741156844beaa6e06cb39d13d4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.187536 3169 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:40.187811 3190 sys_catalog.cc:455] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4b2741156844beaa6e06cb39d13d4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.187938 3190 sys_catalog.cc:458] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:40.188146 3239 mvcc.cc:205] Tried to move back new op lower bound from 7315706675923292160 to 7315706675796873216. Current Snapshot: MvccSnapshot[applied={T|T < 7315706675923292160}]
I20260807 00:06:40.190320 3240 mvcc.cc:205] Tried to move back new op lower bound from 7315706675923292160 to 7315706675796873216. Current Snapshot: MvccSnapshot[applied={T|T < 7315706675923292160}]
I20260807 00:06:40.195362 3169 sys_catalog.cc:455] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b4b2741156844beaa6e06cb39d13d4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.195573 3169 sys_catalog.cc:458] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6 [sys.catalog]: This master's current role is: LEADER
I20260807 00:06:40.197248 3221 catalog_manager.cc:1347] Generated new cluster ID: 2eda4a52effb4a7d81c42b1d2b517ba0
I20260807 00:06:40.197328 3221 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260807 00:06:40.197391 3224 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b4b2741156844beaa6e06cb39d13d4f6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "b4b2741156844beaa6e06cb39d13d4f6" member_type: VOTER last_known_addr { host: "127.30.204.126" port: 45209 } } peers { permanent_uuid: "d14b07121e174defb736cddd1626ac0b" member_type: VOTER last_known_addr { host: "127.30.204.125" port: 33131 } } peers { permanent_uuid: "6ee2f5fe5e92432480c16a6c83c2f7c0" member_type: VOTER last_known_addr { host: "127.30.204.124" port: 37455 } } }
I20260807 00:06:40.197618 3224 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0 [sys.catalog]: This master's current role is: FOLLOWER
I20260807 00:06:40.230324 3221 catalog_manager.cc:1370] Generated new certificate authority record
I20260807 00:06:40.231901 3221 catalog_manager.cc:1504] Loading token signing keys...
I20260807 00:06:40.242632 3221 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b4b2741156844beaa6e06cb39d13d4f6: Generated new TSK 0
I20260807 00:06:40.243795 3221 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260807 00:06:40.284238 2938 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.281154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53670 (local address 127.30.204.126:45209)
0807 00:06:40.281311 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.281317 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.282855 (+ 1538us) server_negotiation.cc:407] Connection header received
0807 00:06:40.283110 (+ 255us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.283114 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.283176 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.283237 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.284058 (+ 821us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:53670: BlockingRecv error: recv got EOF from 127.0.0.1:53670 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 00:06:40.285931 3163 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.282181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42554 (local address 127.30.204.124:37455)
0807 00:06:40.282347 (+ 166us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.282352 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.284844 (+ 2492us) server_negotiation.cc:407] Connection header received
0807 00:06:40.285064 (+ 220us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.285069 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.285130 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.285189 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.285783 (+ 594us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42554: BlockingRecv error: recv got EOF from 127.0.0.1:42554 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
W20260807 00:06:40.288466 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.286935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42944 (local address 127.30.204.125:33131)
0807 00:06:40.287089 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.287094 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.287362 (+ 268us) server_negotiation.cc:407] Connection header received
0807 00:06:40.287598 (+ 236us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.287602 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.287657 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.287712 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.288325 (+ 613us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42944: BlockingRecv error: recv got EOF from 127.0.0.1:42944 (error 108)
Metrics: {"server-negotiator.queue_time_us":50}
W20260807 00:06:40.293819 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.291953 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42948 (local address 127.30.204.125:33131)
0807 00:06:40.292081 (+ 128us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.292086 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.292681 (+ 595us) server_negotiation.cc:407] Connection header received
0807 00:06:40.292933 (+ 252us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.292937 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.293039 (+ 102us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.293100 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.293695 (+ 595us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42948: BlockingRecv error: recv got EOF from 127.0.0.1:42948 (error 108)
Metrics: {"server-negotiator.queue_time_us":47}
W20260807 00:06:40.299950 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.298076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42958 (local address 127.30.204.125:33131)
0807 00:06:40.298207 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.298212 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.298797 (+ 585us) server_negotiation.cc:407] Connection header received
0807 00:06:40.299030 (+ 233us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.299034 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.299094 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.299153 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.299816 (+ 663us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42958: BlockingRecv error: recv got EOF from 127.0.0.1:42958 (error 108)
Metrics: {"server-negotiator.queue_time_us":54}
W20260807 00:06:40.306878 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.304974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42970 (local address 127.30.204.125:33131)
0807 00:06:40.305117 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.305123 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.305730 (+ 607us) server_negotiation.cc:407] Connection header received
0807 00:06:40.305973 (+ 243us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.305977 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.306051 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.306110 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.306729 (+ 619us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42970: BlockingRecv error: recv got EOF from 127.0.0.1:42970 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
W20260807 00:06:40.316486 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.314279 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42986 (local address 127.30.204.125:33131)
0807 00:06:40.314458 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.314464 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.315169 (+ 705us) server_negotiation.cc:407] Connection header received
0807 00:06:40.315451 (+ 282us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.315503 (+ 52us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.315580 (+ 77us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.315649 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.316343 (+ 694us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42986: BlockingRecv error: recv got EOF from 127.0.0.1:42986 (error 108)
Metrics: {"server-negotiator.queue_time_us":94}
W20260807 00:06:40.333290 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.327891 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42996 (local address 127.30.204.125:33131)
0807 00:06:40.328048 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.328053 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.331943 (+ 3890us) server_negotiation.cc:407] Connection header received
0807 00:06:40.332243 (+ 300us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.332247 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.332317 (+ 70us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.332382 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.333143 (+ 761us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:42996: BlockingRecv error: recv got EOF from 127.0.0.1:42996 (error 108)
Metrics: {"server-negotiator.queue_time_us":57}
W20260807 00:06:40.355278 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.353056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43000 (local address 127.30.204.125:33131)
0807 00:06:40.353213 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.353219 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.353928 (+ 709us) server_negotiation.cc:407] Connection header received
0807 00:06:40.354224 (+ 296us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.354238 (+ 14us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.354313 (+ 75us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.354382 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.355128 (+ 746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43000: BlockingRecv error: recv got EOF from 127.0.0.1:43000 (error 108)
Metrics: {"server-negotiator.queue_time_us":65}
W20260807 00:06:40.391570 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.390295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43006 (local address 127.30.204.125:33131)
0807 00:06:40.390443 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.390447 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.390619 (+ 172us) server_negotiation.cc:407] Connection header received
0807 00:06:40.390785 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.390787 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.390838 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.390908 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.391406 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43006: BlockingRecv error: recv got EOF from 127.0.0.1:43006 (error 108)
Metrics: {"server-negotiator.queue_time_us":48}
W20260807 00:06:40.459270 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.458060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43016 (local address 127.30.204.125:33131)
0807 00:06:40.458230 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.458235 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.458436 (+ 201us) server_negotiation.cc:407] Connection header received
0807 00:06:40.458576 (+ 140us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.458579 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.458634 (+ 55us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.458706 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.459140 (+ 434us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43016: BlockingRecv error: recv got EOF from 127.0.0.1:43016 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
W20260807 00:06:40.590991 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.589630 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43032 (local address 127.30.204.125:33131)
0807 00:06:40.589824 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.589830 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.589957 (+ 127us) server_negotiation.cc:407] Connection header received
0807 00:06:40.590149 (+ 192us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.590152 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.590225 (+ 73us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.590318 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.590880 (+ 562us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43032: BlockingRecv error: recv got EOF from 127.0.0.1:43032 (error 108)
Metrics: {"server-negotiator.queue_time_us":77}
W20260807 00:06:40.850755 3055 negotiation.cc:337] Failed RPC negotiation. Trace:
0807 00:06:40.849572 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43034 (local address 127.30.204.125:33131)
0807 00:06:40.849708 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:40.849712 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:40.849827 (+ 115us) server_negotiation.cc:407] Connection header received
0807 00:06:40.849940 (+ 113us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:40.849942 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:40.849996 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:40.850074 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0807 00:06:40.850635 (+ 561us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:43034: BlockingRecv error: recv got EOF from 127.0.0.1:43034 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
I20260807 00:06:41.140336 3206 catalog_manager.cc:1259] Loaded cluster ID: 2eda4a52effb4a7d81c42b1d2b517ba0
I20260807 00:06:41.140421 3206 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: loading cluster ID for follower catalog manager: success
I20260807 00:06:41.142123 3206 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: acquiring CA information for follower catalog manager: success
I20260807 00:06:41.143363 3206 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d14b07121e174defb736cddd1626ac0b: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:06:41.176234 3236 catalog_manager.cc:1259] Loaded cluster ID: 2eda4a52effb4a7d81c42b1d2b517ba0
I20260807 00:06:41.176328 3236 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: loading cluster ID for follower catalog manager: success
I20260807 00:06:41.178037 3236 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: acquiring CA information for follower catalog manager: success
I20260807 00:06:41.179255 3236 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6ee2f5fe5e92432480c16a6c83c2f7c0: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260807 00:06:41.369381 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:41.365075 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43046 (local address 127.30.204.125:33131)
0807 00:06:41.365409 (+ 334us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:41.365413 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:41.365432 (+ 19us) server_negotiation.cc:407] Connection header received
0807 00:06:41.365525 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:41.365527 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:41.365685 (+ 158us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:41.365841 (+ 156us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:41.366134 (+ 293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.366688 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.367751 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.367997 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.368084 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:41.368457 (+ 373us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:41.368933 (+ 476us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:41.369033 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:41.369147 (+ 114us) server_negotiation.cc:299] Negotiation successful
0807 00:06:41.369209 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":247,"thread_start_us":86,"threads_started":1}
I20260807 00:06:41.376758 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:41.372794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42558 (local address 127.30.204.124:37455)
0807 00:06:41.373047 (+ 253us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:41.373051 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:41.373066 (+ 15us) server_negotiation.cc:407] Connection header received
0807 00:06:41.373191 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:41.373193 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:41.373318 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:41.373427 (+ 109us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:41.373663 (+ 236us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.374198 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.375080 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.375371 (+ 291us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.375510 (+ 139us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:41.375824 (+ 314us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:41.376317 (+ 493us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:41.376437 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:41.376523 (+ 86us) server_negotiation.cc:299] Negotiation successful
0807 00:06:41.376590 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171,"thread_start_us":70,"threads_started":1}
I20260807 00:06:41.384074 3348 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:41.380347 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43048 (local address 127.30.204.125:33131)
0807 00:06:41.380463 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:41.380467 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:41.380478 (+ 11us) server_negotiation.cc:407] Connection header received
0807 00:06:41.380611 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:41.380614 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:41.380675 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:41.380767 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:41.381009 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.381636 (+ 627us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.382730 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.383071 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.383179 (+ 108us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:41.383394 (+ 215us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:41.383508 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:41.383570 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:41.383868 (+ 298us) server_negotiation.cc:299] Negotiation successful
0807 00:06:41.383933 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260807 00:06:41.384742 3367 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:41.380000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53676 (local address 127.30.204.126:45209)
0807 00:06:41.380274 (+ 274us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:41.380280 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:41.380428 (+ 148us) server_negotiation.cc:407] Connection header received
0807 00:06:41.380546 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:41.380550 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:41.380741 (+ 191us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:41.380870 (+ 129us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:41.381200 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.382012 (+ 812us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.383067 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.383308 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.383393 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:41.383758 (+ 365us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:41.384295 (+ 537us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:41.384390 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:41.384518 (+ 128us) server_negotiation.cc:299] Negotiation successful
0807 00:06:41.384578 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":78,"threads_started":1}
I20260807 00:06:41.384804 3357 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0807 00:06:41.380110 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42566 (local address 127.30.204.124:37455)
0807 00:06:41.380374 (+ 264us) server_negotiation.cc:206] Beginning negotiation
0807 00:06:41.380379 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0807 00:06:41.380854 (+ 475us) server_negotiation.cc:407] Connection header received
0807 00:06:41.380903 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0807 00:06:41.380906 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0807 00:06:41.380952 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0807 00:06:41.381014 (+ 62us) server_negotiation.cc:226] Negotiated authn=TOKEN
0807 00:06:41.381368 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.382099 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.383507 (+ 1408us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0807 00:06:41.383786 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0807 00:06:41.383874 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0807 00:06:41.384088 (+ 214us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0807 00:06:41.384165 (+ 77us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0807 00:06:41.384234 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0807 00:06:41.384614 (+ 380us) server_negotiation.cc:299] Negotiation successful
0807 00:06:41.384673 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":182}
I20260807 00:06:41.388044 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2824
I20260807 00:06:41.401818 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 2944
I20260807 00:06:41.412267 31537 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskl_Zz3S/build/debug/bin/kudu with pid 3056
2026-08-07T00:06:41Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (3335 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (6750 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (299981 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST